Cache processing of bulk lookups and de-dup concurrent requests

- Fix #93
- Fix #98

.gitignore

@@ -7,6 +7,7 @@ out/
 .idea/
 
 # Local dev config
+/mxisd.yaml
 /application.yaml
 
 # Local dev storage

.travis.yml

@@ -1,4 +1,8 @@
-language: groovy
+language: java
-
+before_cache:
-jdk:
+  - rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock
-  - oraclejdk8
+  - rm -fr $HOME/.gradle/caches/*/plugin-resolution/
+cache:
+  directories:
+    - $HOME/.gradle/caches/
+    - $HOME/.gradle/wrapper/

Dockerfile

@@ -1,11 +1,18 @@
 FROM openjdk:8-jre-alpine
 
+RUN apk update && apk add bash && rm -rf /var/lib/apk/* /var/cache/apk/*
+
 VOLUME /etc/mxisd
 VOLUME /var/mxisd
 EXPOSE 8090
 
-ADD build/libs/mxisd.jar /mxisd.jar
-ADD src/docker/start.sh /start.sh
-
 ENV JAVA_OPTS=""
+ENV CONF_FILE_PATH="/etc/mxisd/mxisd.yaml"
+ENV SIGN_KEY_PATH="/var/mxisd/sign.key"
+ENV SQLITE_DATABASE_PATH="/var/mxisd/mxisd.db"
+
 CMD [ "/start.sh" ]
+
+ADD src/docker/start.sh /start.sh
+ADD src/script/mxisd /app/mxisd
+ADD build/libs/mxisd.jar /app/mxisd.jar

README.md

@@ -1,217 +1,111 @@
-mxisd - Federated Matrix Identity Server Daemon
+mxisd - Federated Matrix Identity Server
------
+----------------------------------------
-![Travis-CI build status](https://travis-ci.org/kamax-io/mxisd.svg?branch=master)  
+![Travis-CI build status](https://travis-ci.org/kamax-matrix/mxisd.svg?branch=master)  
 
-[Overview](#overview) | [Features](#features) | [Lookup process](#lookup-process) | [Packages](#packages) | 
+- [Overview](#overview)
-[From source](#from-source) | [Configuration](#configuration) | [Network Discovery](#network-discovery) | 
+- [Features](#features)
-[Integration](#integration) | [Support](#support)
+- [Use cases](#use-cases)
+- [Getting Started](#getting-started)
+- [Support](#support)
+- [Contribute](#contribute)
+- [Powered by mxisd](#powered-by-mxisd)
+- [FAQ](#faq)
+- [Contact](#contact)
 
 # Overview
-mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures.
+mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with [enhanced features](#features).
+As an enhanced Identity service, it implements the [Matrix Identity service API](https://kamax.io/matrix/api/identity_service/unstable.html)
+and several [extra features](#features) that greatly enhance user experience within Matrix.
+It is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built in a
+single coherent product.
   
-mxisd uses a cascading lookup model which performs lookup from a more authoritative to a less authoritative source, usually doing:
+mxisd is specifically designed to connect to an existing on-premise Identity store (AD/Samba/LDAP, SQL Database,
-- Local identity stores: LDAP, etc.
+Web services/app, etc.) and ease the integration of a Matrix infrastructure within an existing one.
-- Federated identity stores: another Identity Server in charge of a specific domain, if applicable
-- Configured identity stores: another Identity Server specifically configured, if part of some sort of group trust
-- Root identity store: vector.im/matrix.org central Identity Servers
 
-mxisd provides an alternative to [sydent](https://github.com/matrix-org/sydent), while still connecting to the vector.im and matrix.org Identity servers, 
+The core principle of mxisd is to map between Matrix IDs and 3PIDs (Third-Party IDentifiers) for the Homeserver and its
-by implementing the [Matrix Identity Service specification](https://matrix.org/docs/spec/identity_service/unstable.html).
+users. 3PIDs can be anything that uniquely and globally identify a user, like:
+- Email address
+- Phone number
+- Skype/Live ID
+- Twitter handle
+- Facebook ID
 
-mxisd only aims to support workflows that do NOT break federation or basic lookup processes of the Matrix ecosystem.
+If you are unfamiliar with the Identity vocabulary and concepts in Matrix, **please read this [introduction](docs/concepts.md)**.
 
 # Features
-- Single lookup of 3PID (E-mail, phone number, etc.) by the Matrix Client or Homeserver.
+[Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://kamax.io/matrix/api/identity_service/unstable.html#general-principles):
-- Bulk lookups when trying to find possible matches within contacts in Android and iOS clients.
+- Search for people by 3PID using its own Identity stores
-- Bind of 3PID by a Matrix user within a Matrix client.
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#association-lookup))
-- Support of invitation to rooms by e-mail with e-mail notification to invitee.
+- Invite people to rooms by 3PID using its own Identity stores, with notifications to the invitee (Email, SMS, etc.)
-- Authentication support in [synapse](https://github.com/matrix-org/synapse) via the [REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth).
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#post-matrix-identity-api-v1-store-invite))
+- Allow users to add 3PIDs to their settings/profile
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#establishing-associations))
+- Register accounts on your Homeserver with 3PIDs
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#establishing-associations))
 
-In the pipe:
+As an enhanced Identity service:
-- Support to proxy 3PID bindings in user profile to the central Matrix.org servers
+- [Federation](docs/features/federation.md): Use a recursive lookup mechanism when searching and inviting people by 3PID,
+  allowing to fetch data from:
+  - Own Identity store(s)
+  - Federated Identity servers, if applicable to the 3PID
+  - Arbitrary Identity servers
+  - Central Matrix Identity servers
+- [Session Control](docs/threepids/session/session.md): Extensive control of where 3PIDs are transmitted so they are not
+  leaked publicly by users
+- [Authentication](docs/features/authentication.md): Use your Identity stores to perform authentication in [synapse](https://github.com/matrix-org/synapse)
+  via the [REST password provider](https://github.com/kamax-io/matrix-synapse-rest-auth)
+- [Directory search](docs/features/directory.md) which allows you to search for users within your organisation,
+  even without prior contact within Matrix using arbitrary search terms
+- [Auto-fill of user profile](docs/features/authentication.md#profile-auto-fill) (Display name, 3PIDs)
+- [Bridge Integration](docs/features/bridge-integration.md): Automatically bridge users without a published Matrix ID
 
-# Lookup Process
+# Use cases
-Default Lookup strategy will use a priority order and a configurable recursive/local type of request.
+- Use your existing Identity stores, do not duplicate your users information
+- Auto-fill user profiles with relevant information
+- As an organisation, stay in control of your data so it is not published to other servers by default where they
+  currently **cannot be removed**
+- Users can directly find each other using whatever attribute is relevant within your Identity store
+- Federate your Identity server so you can discover others and/or others can discover you
 
-## E-mail
+# Getting started
-Given the 3PID `john.doe@example.org`, the following will be performed until a mapping is found:
+See the [dedicated document](docs/getting-started.md)
-- LDAP: lookup the Matrix ID (partial or complete) from a configurable attribute using a dedicated query.
-- DNS: lookup another Identity Server using the domain part of an e-mail and:
-  - Look for a SRV record under `_matrix-identity._tcp.example.org`
-  - Lookup using the base domain name `example.org`
-- Forwarder: Proxy the request to other configurable identity servers.
-
-## Phone number
-Given the phone number `+123456789`, the following lookup logic will be performed:
-- LDAP: lookup the Matrix ID (partial or complete) from a configurable attribute using a dedicated query.
-- Forwarder: Proxy the request to other configurable identity servers.
-
-# Packages
-See [releases]((https://github.com/kamax-io/mxisd/releases)) for native installers of supported systems.  
-If none is available, please use other packages or build from source.
-
-## Debian
-### Download
-See the [releases section](https://github.com/kamax-io/mxisd/releases).
-
-### Configure and run
-After installation:
-1. Copy the sample config file `/etc/mxisd/mxisd-sample.yaml` to `/etc/mxisd/mxisd.yaml`
-2. [Configure](#configuration)
-3. Start the service: `sudo systemctl start mxisd`
-
-### From source
-Requirements:
-- fakeroot
-- dpkg-deb
-
-Run:
-```
-./gradlew buildDeb 
-```
-You will find the debian package in `build/dist`
-
-## Docker
-```
-docker pull kamax/mxisd
-```
-
-For more info, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)
-### From source
-[Build mxisd](#build) then build the docker image:
-```
-./gradlew dockerBuild
-```
-You can run a container of the given image and test it with the following command (adapt volumes host paths):
-```
-docker run -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd:latest-dev
-```
-
-# From Source
-## Requirements
-- JDK 1.8
-
-## Build
-```
-git clone https://github.com/kamax-io/mxisd.git
-cd mxisd
-./gradlew build
-```
-then see the [Configuration](#configuration) section.
-
-## Test build
-Start the server in foreground to validate the build:
-```
-java -jar build/libs/mxisd.jar
-```
-
-Ensure the signing key is available:
-```
-$ curl http://localhost:8090/_matrix/identity/api/v1/pubkey/ed25519:0
-{"public_key":"..."}
-```
-
-Test basic recursive lookup (requires Internet connection with access to TCP 443):
-```
-$ curl 'http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=mxisd-lookup-test@kamax.io'
-{"address":"mxisd-lookup-test@kamax.io","medium":"email","mxid":"@mxisd-lookup-test:kamax.io",...}
-```
-
-
-If you enabled LDAP, you can also validate your config with a similar request after replacing the `address` value with something present within your LDAP
-```
-curl "http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=john.doe@example.org"
-```
-
-If you plan on testing the integration with a homeserver, you will need to run an HTTPS reverse proxy in front of it
-as the reference Home Server implementation [synapse](https://github.com/matrix-org/synapse) requires a HTTPS connection
-to an ID server.  
-See the [Integration section](https://github.com/kamax-io/mxisd#integration) for more details.
-
-## Install
-After [building](#build) the software, run all the following commands as `root` or using `sudo`
-1. Prepare files and directories:
-```
-# Create a dedicated user
-useradd -r mxisd
-
-# Create bin directory
-mkdir /opt/mxisd
-
-# Create config directory and set ownership
-mkdir /etc/mxisd
-chown mxisd /etc/mxisd
-
-# Create data directory and set ownership
-mkdir /var/opt/mxisd
-chown mxisd /var/opt/mxisd
-
-# Copy <repo root>/build/libs/mxisd.jar to bin directory
-cp ./build/libs/mxisd.jar /opt/mxisd/
-chown mxisd /opt/mxisd/mxisd.jar
-chmod a+x /opt/mxisd/mxisd.jar
-
-# Create symlink for easy exec
-ln -s /opt/mxisd/mxisd.jar /usr/bin/mxisd
-```
-
-2. Copy the config file created earlier `./application.example.yaml` to `/etc/mxisd/mxisd.yaml`
-3. [Configure](#configuration)
-4. Copy `<repo root>/src/systemd/mxisd.service` to `/etc/systemd/system/` and edit if needed
-5. Enable service for auto-startup
-```
-systemctl enable mxisd
-```
-6. Start mxisd
-```
-systemctl start mxisd
-```
-
-# Configuration
-After following the specific instructions to create a config file from the sample:
-1. Set the `matrix.domain` value to the domain value used in your Home Server configuration
-2. Set an absolute location for the signing keys using `key.path`
-3. Set a location for the default SQLite persistence using `storage.provider.sqlite.database`
-4. Configure the E-mail invite sender with items starting in `invite.sender.email`
-
-In case your IS public domain does not match your Matrix domain, see `server.name` and `server.publicUrl` 
-config items.
-
-If you want to use the LDAP backend:
-1. Enable it with `ldap.enabled`
-2. Configure connection options using items starting in `ldap.connection`
-3. You may want to valid default values for `ldap.attribute` items
-
-
-# Network Discovery
-To allow other federated Identity Server to reach yours, the same algorithm used for Homeservers takes place:
-1. Check for the appropriate DNS SRV record
-2. If not found, use the base domain
- 
-If your Identity Server public hostname does not match your Matrix domain, configure the following DNS SRV entry 
-and replace `matrix.example.com` by your Identity server public hostname - **Make sure to end with a final dot!**
-```
-_matrix-identity._tcp.example.com. 3600 IN SRV 10 0 443 matrix.example.com.
-``` 
-This would only apply for 3PID that are DNS-based, like e-mails. For anything else, like phone numbers, no federation 
-is currently possible.  
-
-The port must be HTTPS capable. Typically, the port `8090` of mxisd should be behind a reverse proxy which does HTTPS.
-See the [integration section](#integration) for more details.
-
-# Integration
-- [HTTPS and Reverse proxy](https://github.com/kamax-io/mxisd/wiki/HTTPS)
-- [synapse](https://github.com/kamax-io/mxisd/wiki/Homeserver-Integration) as Identity server
-- [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md) 
-as authentication module
 
 # Support
 ## Community
-If you need help, want to report a bug or just say hi, you can reach us on Matrix at 
+Over Matrix: [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) ([Preview](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/))
-[#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) or [directly peek anonymously](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/).
-For more high-level discussion about the Identity Server architecture/API, go to 
-[#matrix-identity:matrix.org](https://matrix.to/#/#matrix-identity:matrix.org)
 
-## Professional
+For more high-level discussion about the Identity Server architecture/API, go to  [#matrix-identity:kamax.io](https://matrix.to/#/#matrix-identity:kamax.io)
-If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open source technologies/products, 
+
-please visit [our website](https://www.kamax.io/) to get in touch with us and get a quote.
+## Commercial
+If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open
+source technologies/products:
+- Visit our [website](https://www.kamax.io/) to get in touch with us and get a quote.
+- Come in our general Matrix room: [#kamax-matrix:kamax.io](https://matrix.to/#/#kamax-matrix:kamax.io)
+
+# Contribute 
+You can contribute as a community member by:
+- Giving us feedback about your usage of mxisd, even if it seems unimportant or if all is working well!
+- Opening issues for any weird behaviour or bug. mxisd should feel natural, let us know if it does not!
+- Helping us improve the documentation: tell us what is good or not good (in an issue or in Matrix), or make a PR with
+changes you feel improve the doc.
+- Contribute code directly: we love contributors! All your contributions will be licensed under AGPLv3.
+- [Donate!](https://liberapay.com/maximusdor/) Any donation is welcome, regardless how small or big, and will directly
+be used for the fixed costs and developer time of mxisd.
+
+You can contribute as an organisation/corporation by:
+- Get a [support contract](#commercial). This is the best way you can help us as it ensures mxisd is
+maintained regularly and you get direct access to the support team.
+- Sponsoring new features or bug fixes. [Get in touch](#contact) so we can discuss it further.
+
+# Powered by mxisd
+The following projects use mxisd under the hood for some or all their features. Check them out!
+- [matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy)
+- [matrix-register-bot](https://github.com/krombel/matrix-register-bot)
+
+# FAQ
+See the [dedicated document](docs/faq.md)
+
+# Contact
+Get in touch via:
+- Matrix: [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io)
+- Email: see our website: [Kamax.io](https://www.kamax.io)

application.example.yaml

@@ -1,414 +0,0 @@
-# Sample configuration file explaining all possible options, their default value and if they are required or not.
-#
-# Any optional configuration item will be prefixed by # (comment character) with the configuration item following
-# directly without any whitespace character.
-# Default values for optional configuration item will also follow such item.
-#
-# Any mandatory configuration item will not be prefixed by # and will also contain a value as example that must be
-# changed. It is advised to re-create a clean config file with only the required configuration item.
-
-#######################
-# Matrix config items #
-#######################
-# Matrix domain, same as the domain configure in your Homeserver configuration.
-#
-# This is used to build the various identifiers for identity, auth and directory.
-matrix.domain: ''
-
-
-
-#######################
-# Server config items #
-#######################
-# Indicate on which port the Identity Server will listen.
-#
-# This is be default an unencrypted port.
-# HTTPS can be configured using Tomcat configuration properties.
-#
-#server.port: 8090
-
-
-# Public hostname of this identity server.
-#
-# This would be typically be the same as your Matrix domain.
-# In case it is not, set this value.
-#
-# This value is used in various signatures within the Matrix protocol and should be a reachable hostname.
-# You can validate by ensuring you see a JSON answer when calling (replace the domain):
-# https://example.org/_matrix/identity/status
-#
-#server.name: 'example.org'
-
-
-# Public URL to reach this identity server
-#
-# This is used with 3PID invites in room and other Homeserver key verification workflow.
-# If left unconfigured, it will be generated from the server name.
-#
-# You should typically set this value if you want to change the public port under which
-# this Identity server is reachable.
-#
-# %SERVER_NAME% placeholder is available to avoid configuration duplication.
-# e.g. 'https://%SERVER_NAME%:8443'
-#
-#server.publicUrl: 'https://example.org'
-
-
-
-#############################
-# Signing keys config items #
-#############################
-# Absolute path for the Identity Server signing key.
-# During testing, /var/tmp/mxisd.key is a possible value
-#
-# For production, use a stable location like:
-#   - /var/opt/mxisd/sign.key
-#   - /var/local/mxisd/sign.key
-#   - /var/lib/mxisd/sign.key
-key.path: '/path/to/sign.key'
-
-
-
-#################################
-# Recurisve lookup config items #
-#################################
-# Configuration items for recursion-type of lookup
-#
-# Lookup access are divided into two types:
-# - Local
-# - Remote
-#
-# This is similar to DNS lookup and recursion and is therefore prone to the same vulnerabilities.
-# By default, only non-public hosts are allowed to perform recursive lookup.
-#
-# This will also prevent very basic endless loops where host A ask host B, which in turn is configured to ask host A,
-# which would then ask host B again, etc.
-
-# Enable recursive lookup globally
-#
-#lookup.recursive.enabled: true
-
-
-# Whitelist of CIDR that will trigger a recursive lookup.
-# The default list includes all private IPv4 address and the IPv6 loopback.
-#
-#lookup.recursive.allowedCidr:
-#  - '127.0.0.0/8'
-#  - '10.0.0.0/8'
-#  - '172.16.0.0/12'
-#  - '192.168.0.0/16'
-#  - '::1/128'
-
-
-# In case no binding is found, query an application server which implements the single lookup end-point
-# to return bridge virtual user that would allow the user to be contacted directly by the said bridge.
-#
-# If a binding is returned, the application server is not expected to sign the message as it is not meant to be
-# reachable from the outside.
-# If a signature is provided, it will be discarded/replaced by this IS implementation (to be implemented).
-#
-# IMPORTANT: This bypass the regular Invite system of the Homeserver. It will be up to the Application Server
-# to handle such invite. Also, if the bridged user were to actually join Matrix later, or if a 3PID binding is found
-# room rights and history would not be transferred, as it would appear as a regular Matrix user to the Homeserver.
-#
-# This configuration is only helpful for Application Services that want to overwrite bridging for 3PID that are
-# handled by the Homeserver. Do not enable unless the Application Server specifically supports it!
-
-# Enable unknown 3PID bridging globally
-#
-#lookup.recursive.bridge.enabled: false
-
-
-# Enable unknown 3PID bridging for hosts that are allowed to perform recursive lookups.
-# Leaving this setting to true is highly recommended in a standard setup, unless this Identity Server
-# is meant to always return a virtual user MXID even for the outside world.
-#
-#lookup.recursive.bridge.recursiveOnly: true
-
-
-# This mechanism can handle the following scenarios:
-#
-# - Single Application Server for all 3PID types: only configure the server value, comment out the rest.
-#
-# - Specific Application Server for some 3PID types, default server for the rest: configure the server value and
-#          each specific 3PID type.
-#
-# - Only specific 3PID types: do not configure the server value or leave it empty/blank, configure each specific
-#          3PID type.
-
-# Default application server to use for all 3PID types. Remove config item or leave empty/blank to disable.
-#
-#lookup.recursive.bridge.server: ''
-
-
-# Configure each 3PID type with a specific application server. Remove config item or leave empty/blank to disable.
-#
-#lookup.recursive.bridge.mappings.email: 'http://localhost:8091'
-#lookup.recursive.bridge.mappings.msisdn: ''
-
-
-
-#####################
-# LDAP config items #
-#####################
-# Global enable/disable switch
-#
-#ldap.enabled: false
-
-
-#### Connection related config items
-# If the connection should be secure
-#
-#ldap.connection.tls: false
-
-
-# Host to connect to
-#
-#ldap.connection.host: 'localhost'
-
-
-# Port to connect to
-#
-#ldap.connection.port: 389
-
-
-# Bind DN for the connection.
-#
-# If Bind DN and password are empty, anonymous authentication is performed
-#
-#ldap.connection.bindDn: 'CN=Matrix Identity Server,CN=Users,DC=example,DC=org'
-
-
-# Bind password for the connection.
-#
-#ldap.connection.bindPassword: 'password'
-
-
-# Base DN used in all queries
-#
-#ldap.connection.baseDn: 'CN=Users,DC=example,DC=org'
-
-
-#### How to map Matrix attributes with LDAP attributes when performing lookup/auth
-#
-# How should we resolve the Matrix ID in case of a match using the attribute.
-#
-# The following type are supported:
-# - uid : the attribute only contains the UID part of the Matrix ID. e.g. 'john.doe' in @john.doe:example.org
-# - mxid : the attribute contains the full Matrix ID - e.g. '@john.doe:example.org'
-#
-#ldap.attribute.uid.type: 'uid'
-
-
-# The attribute containing the binding itself. This value will be used differently depending on the type.
-#
-# /!\ This should match the synapse LDAP Authenticator 'uid' configuration /!\
-#
-# Typical values:
-# - For type 'uid': 'userPrincipalName' or 'uid' or 'saMAccountName'
-# - For type 'mxid', regardless of the directory type, we recommend using 'pager' as it is a standard attribute and
-#   is typically not used.
-#
-#ldap.attribute.uid.value: 'userPrincipalName'
-
-
-# The display name of the user
-#
-#ldap.attribute.name: 'displayName'
-
-
-#### Configuration section relating the authentication of users performed via LDAP.
-#
-# This can be done using the REST Auth module for synapse and pointing it to the identity server.
-# See https://github.com/kamax-io/matrix-synapse-rest-auth
-#
-# During authentication, What to filter potential users by, typically by using a dedicated group.
-# If this value is not set, login check will be performed for all entities within the LDAP
-#
-# Example: (memberOf=CN=Matrix Users,CN=Users,DC=example,DC=org)
-#
-#ldap.auth.filter: ''
-
-
-#### Configuration section relating to identity lookups
-#
-# E-mail query
-#
-#ldap.identity.medium.email: "(|(mailPrimaryAddress=%3pid)(mail=%3pid)(otherMailbox=%3pid))"
-
-
-# Phone numbers query
-#
-# Phone numbers use the MSISDN format: https://en.wikipedia.org/wiki/MSISDN
-# This format does not include international prefix (+ or 00) and therefore has to be put in the query.
-# Adapt this to your needs for each attribute.
-#
-#ldap.identity.medium.msisdn: "(|(telephoneNumber=+%3pid)(mobile=+%3pid)(homePhone=+%3pid)(otherTelephone=+%3pid)(otherMobile=+%3pid)(otherHomePhone=+%3pid))"
-
-
-
-############################
-# SQL Provider config item #
-############################
-#
-# Example configuration to integrate with synapse SQLite DB (default configuration)
-#
-#sql.enabled: true
-#sql.type: 'sqlite'
-#sql.connection: '/var/lib/matrix-synapse/homeserver.db'
-
-
-#
-# Example configuration to integrate with synapse PostgreSQL DB
-#sql.enabled: true
-#sql.type: 'postgresql'
-#sql.connection: '//dnsOrIpToServer/dbName?user=synapseDbUser&password=synapseDbPassword'
-
-
-#
-# Configuration for an arbitrary server with arbitrary driver
-#
-# sql.identity.type possible values:
-# - uid       Returned value is the localpart of the Matrix ID
-# - mxid      Full Matrix ID, including domain
-#
-# sql.identity.query MUST contain a column with label 'uid'
-#
-# If you would like to overwrite the global lookup query for specific medium type,
-# add a config item (see below for example) in the following format
-# sql.identity.medium.theMediumIdYouWant: 'the query'
-
-#sql.enabled: true
-#sql.type: 'jdbcDriverName'
-#sql.connection: '//dnsOrIpToServer/dbName?user=synapseDbUser&password=synapseDbPassword'
-#sql.identity.type: 'mxid'
-#sql.identity.query: 'SELECT raw AS uid FROM table WHERE medium = ? AND address = ?'
-#sql.identity.medium.email: 'SELECT raw AS uid FROM emailTable WHERE address = ?'
-
-
-
-#######################################
-# Lookup queries forward config items #
-#######################################
-# List of forwarders to use to try to match a 3PID.
-#
-# Each server will be tried in the given order, going to the next if no binding was found or an error occurred.
-# These are the current root Identity Servers of the Matrix network.
-#
-#forward.servers:
-#  - "https://matrix.org"
-#  - "https://vector.im"
-
-
-
-#############################
-# 3PID invites config items #
-#############################
-#
-#### E-mail invite sender
-#
-# SMTP host
-invite.sender.email.host: "smtp.example.org"
-
-
-# SMTP port
-invite.sender.email.port: 587
-
-
-# TLS mode for the connection.
-#
-# Possible values:
-#  0    Disable TLS entirely
-#  1    Enable TLS if supported by server
-#  2    Force TLS and fail if not available
-#
-#invite.sender.email.tls: 1
-
-
-# Login for SMTP
-invite.sender.email.login: "matrix-identity@example.org"
-
-
-# Password for the account
-invite.sender.email.password: "ThePassword"
-
-
-# The e-mail to send as. If empty, will be the same as login
-invite.sender.email.email: "matrix-identity@example.org"
-
-
-# The display name used in the e-mail
-#
-#invite.sender.email.name: "mxisd Identity Server"
-
-
-# The E-mail template to use, using built-in template by default
-#
-# The template is expected to be a full e-mail body, including client headers, using MIME and UTF-8 encoding.
-# The following headers will be set by mxisd directly and should not be present in the template:
-# - From
-# - To
-# - Date
-# - Message-Id
-# - X-Mailer
-#
-# The following placeholders are available:
-# - %DOMAIN%                Domain name as per server.name config item
-# - %DOMAIN_PRETTY%         Word capitalize version of the domain. e.g. example.org -> Example.org
-# - %FROM_EMAIL%            Value of this section's email config item
-# - %FROM_NAME%             Value of this section's name config item
-# - %SENDER_ID%             Matrix ID of the invitation sender
-# - %SENDER_NAME%           Display name of the invitation sender, empty if not available
-# - %SENDER_NAME_OR_ID%     Value of %SENDER_NAME% or, if empty, value of %SENDER_ID%
-# - %INVITE_MEDIUM%         Medium of the invite (e.g. email, msisdn)
-# - %INVITE_ADDRESS%        Address used to invite
-# - %ROOM_ID%               ID of the room where the invitation took place
-# - %ROOM_NAME%             Name of the room, empty if not available
-# - %ROOM_NAME_OR_ID%       Value of %ROOM_NAME% or, if empty, value of %ROOM_ID%
-#
-#invite.sender.email.template: "/absolute/path/to/file"
-
-
-
-############################
-# Persistence config items #
-############################
-
-# Configure the storage backend, usually a DB
-# Possible built-in values:
-#   sqlite                      SQLite backend, default
-#
-#storage.backend: 'sqlite'
-
-
-#### Generic SQLite provider config
-#
-# Path to the SQLite DB file, required if SQLite backend is chosen
-#
-# Examples:
-#  - /var/opt/mxisd/mxisd.db
-#  - /var/local/mxisd/mxisd.db
-#  - /var/lib/mxisd/mxisd.db
-#
-storage.provider.sqlite.database: '/path/to/mxisd.db'
-
-
-
-######################
-# DNS-related config #
-######################
-# The domain to overwrite
-#
-#dns.overwrite.homeserver.name: 'example.org'
-
-
-# - 'env' from environment variable specified by value
-# - any other value will use the value as-is as host
-#
-#dns.overwrite.homeserver.type: 'raw'
-
-
-# The value to use, depending on the type.
-# Protocol will always be HTTPS
-#
-#dns.overwrite.homeserver.value: 'localhost:8448'

build.gradle

@@ -1,10 +1,8 @@
-import java.util.regex.Pattern
-
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2017 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -20,10 +18,14 @@ import java.util.regex.Pattern
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-apply plugin: 'groovy'
+import java.util.regex.Pattern
-apply plugin: 'org.springframework.boot'
 
-def confFileName = "application.example.yaml"
+apply plugin: 'java'
+apply plugin: 'application'
+apply plugin: 'com.github.johnrengelman.shadow'
+apply plugin: 'idea'
+
+def confFileName = "mxisd.example.yaml"
 def distDir = "${project.buildDir}/dist"
 
 def debBinPath = "/usr/lib/mxisd"
@@ -31,7 +33,8 @@ def debConfPath = "/etc/mxisd"
 def debDataPath = "/var/lib/mxisd"
 def debSystemdPath = "/etc/systemd/system"
 
-def debConfFileName = "mxisd-sample.yaml"
+def debConfFileName = confFileName
+def debStartScriptFilename = "mxisd"
 
 def debBuildBasePath = "${project.buildDir}/tmp/debian"
 def debBuildDebianPath = "${debBuildBasePath}/DEBIAN"
@@ -41,46 +44,61 @@ def debBuildDataPath = "${debBuildBasePath}${debDataPath}"
 def debBuildSystemdPath = "${debBuildBasePath}${debSystemdPath}"
 
 def dockerImageName = "kamax/mxisd"
-def dockerImageTag = "${dockerImageName}:${gitVersion()}"
+def dockerImageTag = "${dockerImageName}:${mxisdVersion()}"
+
+group = 'io.kamax'
+mainClassName = 'io.kamax.mxisd.MxisdStandaloneExec'
+
+String mxisdVersion() {
+    def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")
+
+    String version = System.getenv('MXISD_BUILD_VERSION')
+    if (version == null || version.size() == 0) {
+        version = gitVersion()
+    }
+    return versionPattern.matcher(version).matches() ? version.substring(1) : version
+}
 
 String gitVersion() {
-    def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")
     ByteArrayOutputStream out = new ByteArrayOutputStream()
     exec {
-        commandLine = [ 'git', 'describe', '--always', '--dirty' ]
+        commandLine = ['git', 'describe', '--tags', '--always', '--dirty']
         standardOutput = out
     }
-    def v = out.toString().replace(System.lineSeparator(), '')
+    return out.toString().replace(System.lineSeparator(), '')
-    return versionPattern.matcher(v).matches() ? v.substring(1) : v
 }
 
 buildscript {
     repositories {
-        mavenCentral()
+        jcenter()
     }
 
     dependencies {
-        classpath 'org.springframework.boot:spring-boot-gradle-plugin:1.5.3.RELEASE'
+        classpath 'com.github.jengelman.gradle.plugins:shadow:4.0.3'
     }
 }
 
 repositories {
+    jcenter()
     maven { url "https://kamax.io/maven/releases/" }
-    mavenCentral()
+    maven { url "https://kamax.io/maven/snapshots/" }
 }
 
 dependencies {
-    // We are a groovy project
+    // Logging
-    compile 'org.codehaus.groovy:groovy-all:2.4.7'
+    compile 'org.slf4j:slf4j-simple:1.7.25'
     
     // Easy file management
     compile 'commons-io:commons-io:2.5'
     
-    // Spring Boot - standalone app
+    // Config management
-    compile 'org.springframework.boot:spring-boot-starter-web:1.5.3.RELEASE'
+    compile 'org.yaml:snakeyaml:1.23'
 
     // Matrix Java SDK
-    compile 'io.kamax:matrix-java-sdk:0.0.2'
+    compile 'io.kamax:matrix-java-sdk:0.0.14-8-g0e57ec6'
+
+    // ORMLite
+    compile 'com.j256.ormlite:ormlite-jdbc:5.0'
 
     // ed25519 handling
     compile 'net.i2p.crypto:eddsa:0.1.0'
@@ -94,58 +112,59 @@ dependencies {
     // HTTP connections
     compile 'org.apache.httpcomponents:httpclient:4.5.3'
 
-    // JSON
-    compile 'com.google.code.gson:gson:2.8.1'
-
     // Phone numbers validation
     compile 'com.googlecode.libphonenumber:libphonenumber:8.7.1'
 
     // E-mail sending
-    compile 'com.sun.mail:javax.mail:1.5.6'
+    compile 'javax.mail:javax.mail-api:1.6.2'
-    compile 'javax.mail:javax.mail-api:1.5.6'
+    compile 'com.sun.mail:javax.mail:1.6.2'
 
     // Google Firebase Authentication backend
     compile 'com.google.firebase:firebase-admin:5.3.0'
 
-    // ORMLite
+    // Connection Pool
-    compile 'com.j256.ormlite:ormlite-jdbc:5.0'
+    compile 'com.mchange:c3p0:0.9.5.2'
 
     // SQLite
     compile 'org.xerial:sqlite-jdbc:3.20.0'
 
     // PostgreSQL
-    compile 'org.postgresql:postgresql:42.1.4'
+    compile 'org.postgresql:postgresql:42.2.5'
+
+    // MariaDB/MySQL
+    compile 'org.mariadb.jdbc:mariadb-java-client:2.1.2'
+
+    // Twilio SDK for SMS
+    compile 'com.twilio.sdk:twilio:7.14.5'
+
+    // SendGrid SDK to send emails from GCE
+    compile 'com.sendgrid:sendgrid-java:2.2.2'
+
+    // ZT-Exec for exec identity store
+    compile 'org.zeroturnaround:zt-exec:1.10'
+
+    // HTTP server
+    compile 'io.undertow:undertow-core:2.0.16.Final'
 
     testCompile 'junit:junit:4.12'
     testCompile 'com.github.tomakehurst:wiremock:2.8.0'
+    testCompile 'com.unboundid:unboundid-ldapsdk:4.0.9'
+    testCompile 'com.icegreen:greenmail:1.5.9'
 }
 
-springBoot {
+shadowJar {
-    executable = true
+    baseName = project.name
-
+    classifier = null
-    embeddedLaunchScriptProperties = [
+    version = null
-            confFolder: "/etc/default"
-    ]
 }
 
-processResources {
+task debBuild(dependsOn: shadowJar) {
     doLast {
-        copy {
+        String debVersion = mxisdVersion()
-            from('build/resources/main/application.yaml') {
+        println "Version for package: ${debVersion}"
-                rename 'application.yaml', 'mxisd.yaml'
-            }
-            into 'build/resources/main'
-        }
-    }
-}
-
-task buildDeb(dependsOn: build) {
-    doLast {
-        def v = gitVersion()
-        println "Version for package: ${v}"
         mkdir distDir
         mkdir debBuildBasePath
-        mkdir "${debBuildBasePath}/DEBIAN"
+        mkdir debBuildDebianPath
         mkdir debBuildBinPath
         mkdir debBuildConfPath
         mkdir debBuildDataPath
@@ -156,10 +175,10 @@ task buildDeb(dependsOn: build) {
             into debBuildBinPath
         }
 
-        ant.chmod(
+        copy {
-            file: "${debBuildBinPath}/mxisd.jar",
+            from "${project.file("src/script/" + debStartScriptFilename)}"
-            perm: 'a+x'
+            into debBuildBinPath
-        )
+        }
 
         copy {
             from(project.file(confFileName)) {
@@ -168,16 +187,16 @@ task buildDeb(dependsOn: build) {
             into debBuildConfPath
         }
 
-        ant.replaceregexp(
+        ant.replaceregexp( // FIXME adapt to new config format
-            file: "${debBuildConfPath}/${debConfFileName}",
+                file: "${debBuildConfPath}/${debConfFileName}",
-            match: "key.path:(.*)",
+                match: "key:\\R  path:(.*)",
-            replace: "key.path: '${debDataPath}/signing.key'"
+                replace: "key:\n  path: '${debDataPath}/signing.key'"
         )
 
-        ant.replaceregexp(
+        ant.replaceregexp( // FIXME adapt to new config format
-            file: "${debBuildConfPath}/${debConfFileName}",
+                file: "${debBuildConfPath}/${debConfFileName}",
-            match: "storage.provider.sqlite.database:(.*)",
+                match: "storage:\\R  provider:\\R    sqlite:\\R      database:(.*)",
-            replace: "storage.provider.sqlite.database: '${debDataPath}/mxisd.db'"
+                replace: "storage:\n  provider:\n    sqlite:\n      database: '${debDataPath}/mxisd.db'"
         )
 
         copy {
@@ -186,25 +205,25 @@ task buildDeb(dependsOn: build) {
         }
 
         ant.replace(
-            file: "${debBuildDebianPath}/control",
+                file: "${debBuildDebianPath}/control",
-            token: 'Version: 0',
+                token: 'Version: 0',
-            value: "Version: ${v}"
+                value: "Version: ${debVersion}"
         )
 
         ant.replace(
-            file: "${debBuildDebianPath}/postinst",
+                file: "${debBuildDebianPath}/postinst",
-            token: '%DEB_DATA_DIR%',
+                token: '%DEB_DATA_DIR%',
-            value: debDataPath
+                value: debDataPath
         )
 
         ant.chmod(
-            file: "${debBuildDebianPath}/postinst",
+                file: "${debBuildDebianPath}/postinst",
-            perm: 'a+x'
+                perm: 'a+x'
         )
 
         ant.chmod(
-            file: "${debBuildDebianPath}/prerm",
+                file: "${debBuildDebianPath}/prerm",
-            perm: 'a+x'
+                perm: 'a+x'
         )
 
         copy {
@@ -214,17 +233,17 @@ task buildDeb(dependsOn: build) {
 
         exec {
             commandLine(
-                'fakeroot',
+                    'fakeroot',
-                'dpkg-deb',
+                    'dpkg-deb',
-                '-b',
+                    '-b',
-                debBuildBasePath,
+                    debBuildBasePath,
-                "${project.buildDir}/dist"
+                    "${project.buildDir}/dist"
             )
         }
     }
 }
 
-task dockerBuild(type: Exec, dependsOn: build) {
+task dockerBuild(type: Exec, dependsOn: shadowJar) {
     commandLine 'docker', 'build', '-t', dockerImageTag, project.rootDir
 
     doLast {

docs/README.md

@@ -0,0 +1,26 @@
+# Table of Contents
+- [Identity Concepts in Matrix](concepts.md)
+- [Getting Started](getting-started.md)
+- [Build from sources](build.md) (Optional)
+- Installation
+  - [Debian package](install/debian.md)
+  - [ArchLinux](install/archlinux.md)
+  - [NixOS](install/nixos.md)
+  - [Docker](install/docker.md)
+  - [From source](install/source.md)
+- [Architecture overview](architecture.md)
+- [Configuration](configure.md)
+- Features
+  - [Authentication](features/authentication.md)
+  - [Directory search](features/directory.md)
+  - [Identity](features/identity.md)
+  - [Federation](features/federation.md)
+  - [Bridge integration](features/bridge-integration.md)
+- [Identity Stores](stores/README.md)
+- Notifications
+  - Handlers
+    - [Basic](threepids/notification/basic-handler.md)
+    - [SendGrid](threepids/notification/sendgrid-handler.md)
+- [Sessions](threepids/session/session.md)
+  - [Views](threepids/session/session-views.md)
+- [FAQ](faq.md)

docs/_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-hacker

docs/architecture.md

@@ -0,0 +1,40 @@
+# Architecture
+## Overview
+### Basic setup with default settings
+```
+ Client
+   |
+TCP 443
+   |   +---------------------+            +---------------------------+
+   +-> | Reverse proxy       |            | Homeserver                |
+       |                     | TCP 8008   |                           |
+       |  /_matrix/* -------------------> | - 3PID invite from client |
+       |                     |            |   |                       |
+       |  /_matrix/identity/ |            |   |                       |
+       +--|------------------+            +---|-----------------------+
+          |                                   |
+          +<---------------------------------<+
+          |
+          |   +-------------------+
+ TCP 8090 +-> | mxisd             |
+              |                   |
+              | - Profile's 3PIDs |
+              | - 3PID Invites    |
+              +-|-----------------+
+                |
+             TCP 443
+                |  +------------------------+
+                |  | Remote Federated       |
+                |  | mxisd servers          |
+                |  |                        |
+                +--> - 3PID Invites         |
+                   +------------------------+
+```
+### With Authentication
+See the [dedicated document](features/authentication.md).
+
+### With Directory
+See the [dedicated document](features/directory.md).
+
+### With Federation
+See the [dedicated document](features/federation.md).

docs/backends/rest.md

@@ -1,169 +0,0 @@
-# REST backend
-The REST backend allows you to query arbitrary REST JSON endpoints as backends for the following flows:
-- Identity lookup
-- Authentication
-
-## Configuration
-| Key                            | Default                               | Description                                          |
----------------------------------|---------------------------------------|------------------------------------------------------|
-| rest.enabled                   | false                                 | Globally enable/disable the REST backend             |
-| rest.host                      | *empty*                               | Default base URL to use for the different endpoints. |
-| rest.endpoints.auth            | /_mxisd/identity/api/v1/auth          | Endpoint to validate credentials                     |
-| rest.endpoints.identity.single | /_mxisd/identity/api/v1/lookup/single | Endpoint to lookup a single 3PID                     |
-| rest.endpoints.identity.bulk   | /_mxisd/identity/api/v1/lookup/bulk   | Endpoint to lookup a list of 3PID                    |
-
-Endpoint values can handle two formats:
-- URL Path starting with `/` that gets happened to the `rest.host`
-- Full URL, if you want each endpoint to go to a specific server/protocol/port
-
-`rest.host` is only mandatory if at least one endpoint is not a full URL.
-
-## Endpoints
-### Authenticate
-Configured with `rest.endpoints.auth`
-
-HTTP method: `POST`  
-Encoding: JSON UTF-8
-  
-#### Request Body
-```
-{
-  "auth": {
-    "mxid": "@john.doe:example.org",
-    "localpart": "john.doe",
-    "domain": "example.org",
-    "password": "passwordOfTheUser"
-  }
-}
-```
-
-#### Response Body
-If the authentication fails:
-```
-{
-  "auth": {
-    "success": false
-  }
-}
-```
-
-If the authentication succeed:
-- `auth.id` supported values: `localpart`, `mxid`
-- `auth.profile` and any sub-member are all optional
-```
-{
-  "auth": {
-    "success": true,
-    "id": {
-      "type": "localpart",
-      "value": "john"
-    },
-    "profile": {
-      "display_name": "John Doe",
-      "three_pids": [
-        {
-          "medium": "email",
-          "address": "john.doe@example.org"
-        },
-        {
-          "medium": "msisdn",
-          "address": "123456789"
-        }
-      ]
-    }
-  }
-}
-```
-
-### Lookup
-#### Single
-Configured with `rest.endpoints.identity.single`
-
-HTTP method: `POST`  
-Encoding: JSON UTF-8  
-  
-#### Request Body
-```
-{
-  "lookup": {
-    "medium": "email",
-    "address": "john.doe@example.org"
-  }
-}
-```
-
-#### Response Body
-If a match was found:
-- `lookup.id.type` supported values: `localpart`, `mxid`
-```
-{
-  "lookup": {
-    "medium": "email",
-    "address": "john.doe@example.org",
-    "id": {
-      "type": "mxid",
-      "value": "@john:example.org"
-    }
-  }
-}
-```
-
-If no match was found:
-```
-{}
-```
-
-#### Bulk
-Configured with `rest.endpoints.identity.bulk`
-
-HTTP method: `POST`  
-Encoding: JSON UTF-8  
-  
-#### Request Body
-```
-{
-  "lookup": [
-    {
-      "medium": "email",
-      "address": "john.doe@example.org"
-    },
-    {
-      "medium": "msisdn",
-      "address": "123456789"
-    }
-  ]
-}
-```
-
-#### Response Body
-For all entries where a match was found:
-- `lookup[].id.type` supported values: `localpart`, `mxid`
-```
-{
-  "lookup": [
-    {
-      "medium": "email",
-      "address": "john.doe@example.org",
-      "id": {
-        "type": "localpart",
-        "value": "john"
-      }
-    },
-    {
-      "medium": "msisdn",
-      "address": "123456789",
-      "id": {
-        "type": "mxid",
-        "value": "@jane:example.org"
-      }
-    }
-  ]
-}
-```
-
-If no match was found:
-```
-{
-  "lookup": []
-}
-```

docs/build.md

@@ -0,0 +1,74 @@
+# From source
+- [Binaries](#binaries)
+  - [Requirements](#requirements)
+  - [Build](#build)
+- [Debian package](#debian-package)
+- [Docker image](#docker-image)
+- [Next steps](#next-steps)
+
+## Binaries
+### Requirements
+- JDK 1.8
+
+### Build
+```bash
+git clone https://github.com/kamax-matrix/mxisd.git
+cd mxisd
+./gradlew build
+```
+
+Create a new configuration file by coping `mxisd.example.yaml` to `mxisd.yaml` and edit to your needs.  
+For advanced configuration, see the [Configure section](configure.md).
+
+Start the server in foreground to validate the build and configuration:
+```bash
+java -jar build/libs/mxisd.jar
+```
+
+Ensure the signing key is available:
+```bash
+$ curl 'http://localhost:8090/_matrix/identity/api/v1/pubkey/ed25519:0'
+
+{"public_key":"..."}
+```
+
+Test basic recursive lookup (requires Internet connection with access to TCP 443):
+```bash
+$ curl 'http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=mxisd-federation-test@kamax.io'
+
+{"address":"mxisd-federation-test@kamax.io","medium":"email","mxid":"@mxisd-lookup-test:kamax.io",...}
+```
+
+If you enabled LDAP, you can also validate your config with a similar request after replacing the `address` value with
+something present within your LDAP
+```bash
+curl 'http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=john.doe@example.org'
+```
+
+If you plan on testing the integration with a homeserver, you will need to run an HTTPS reverse proxy in front of it
+as the reference Home Server implementation [synapse](https://github.com/matrix-org/synapse) requires a HTTPS connection
+to an ID server.  
+
+Next step: [Install your compiled binaries](install/source.md)
+
+## Debian package
+Requirements:
+- fakeroot
+- dpkg-deb
+
+[Build mxisd](#build) then:
+```bash
+./gradlew debBuild
+```
+You will find the debian package in `build/dist`.  
+Then follow the instruction in the [Debian package](install/debian.md) document.
+
+## Docker image
+[Build mxisd](#build) then:
+```bash
+./gradlew dockerBuild
+```
+Then follow the instructions in the [Docker install](install/docker.md#configure) document.
+
+## Next steps
+- [Integrate with your infrastructure](getting-started.md#integrate)

docs/concepts.md

@@ -0,0 +1,43 @@
+# Concepts
+- [Matrix](#matrix)
+- [mxisd](#mxisd)
+
+## Matrix
+The following concepts are part of the Matrix ecosystem and specification.
+
+### 3PID
+`3PID` stands for Third-Party Identifier.  
+It is also commonly written:
+- `3pid`
+- `tpid`
+
+A 3PID is a globally unique canonical identifier which is made of:
+- Medium, which describes what network it belongs to (Email, Phone, Twitter, Discord, etc.)
+- Address, the actual value people typically use on a daily basis.
+
+mxisd core mission is to map those identifiers to Matrix User IDs.
+
+### Homeserver
+Where a user **account and data** are stored.
+
+### Identity server
+An Identity server:
+- Does lookup of 3PIDs to User Matrix IDs.
+- Does validate 3PIDs ownership, typically by sending a code that the user has to enter in an application/on a website.
+- Does send notifications about room invites where no Matrix User ID could be found for the invitee.
+
+An Identity server:
+- **DOES NOT** store user accounts.
+- **DOES NOT** store user data.
+- **DOES NOT** allow migration of user account and/or data between homeservers. 
+
+### 3PID session
+The fact to validate a 3PID (email, phone number, etc.) via the introduction of a token which was sent to the 3PID address.
+
+## mxisd
+The following concepts are specific to mxisd.
+
+### Identity store
+Where your user accounts and 3PID mappings are stored.
+
+mxisd itself **DOES NOT STORE** user accounts or 3PID mappings.

docs/configure.md

@@ -0,0 +1,86 @@
+# Configuration
+- [Concepts](#concepts)
+  - [Syntax](#syntax)
+- [Matrix](#matrix)
+- [Server](#server)
+- [Storage](#storage)
+- [Identity stores](#identity-stores)
+- [3PID Validation sessions](#3pid-validation-sessions)
+- [Notifications](#notifications)
+
+## Concepts
+### Syntax
+The configuration file is [YAML](http://yaml.org/) based:
+```yaml
+my:
+  config:
+    item: 'value'
+
+```
+
+When referencing keys in all documents, a property-like shorthand will be used. The shorthand for the above example would be `my.config.item`
+
+## Matrix
+`matrix.domain`
+Matrix domain name, same as the Homeserver, used to build appropriate Matrix IDs |
+
+---
+
+`matrix.identity.servers`
+Namespace to create arbitrary list of Identity servers, usable in other parts of the configuration |
+
+Example:
+```yaml
+matrix:
+  identity:
+    servers:
+      myOtherServers:
+        - 'https://other1.example.org'
+        - 'https://other2.example.org'
+```
+Create a list under the label `myOtherServers` containing two Identity servers: `https://other1.example.org` and `https://other2.example.org`.
+
+## Server
+- `server.name`: Public hostname of mxisd, if different from the Matrix domain.
+- `server.port`: HTTP port to listen on (unencrypted)
+- `server.publicUrl`: Defaults to `https://{server.name}`
+
+## Storage
+### SQLite
+`storage.provider.sqlite.database`: Absolute location of the SQLite database
+
+## Identity stores
+See the [Identity stores](stores/README.md) for specific configuration
+
+## 3PID Validation sessions
+See the dedicated documents:
+- [Flow](threepids/session/session.md)
+- [Branding](threepids/session/session-views.md)
+
+## Notifications
+- `notification.handler.<3PID medium>`: Handler to use for the given 3PID medium. Repeatable.
+
+Example:
+```yaml
+notification:
+  handler:
+    email: 'sendgrid'
+    msisdn: 'raw'
+```
+- Emails notifications would use the `sendgrid` handler, which define its own configuration under `notification.handlers.sendgrid`
+- Phone notification would use the `raw` handler, basic default built-in handler in mxisd
+
+### Handlers
+- `notification.handers.<handler ID>`: Handler-specific configuration for the given handler ID. Repeatable.
+
+Example:
+```yaml
+notification:
+  handlers:
+    raw: ...
+    sendgrid: ...
+```
+
+Built-in:
+- [Raw](threepids/notification/basic-handler.md)
+- [SendGrid](threepids/notification/sendgrid-handler.md)

docs/faq.md

@@ -0,0 +1,87 @@
+# Frequently Asked Questions
+### This is all very complicated and I'm getting confused with all the words, concepts and diagrams - Help!
+Matrix is still a very young protocol and there are a whole lot of rough edges.  
+Identity in Matrix is one of the most difficult topic, mainly as it has not received much love in the past years.
+
+We have tried our best to put together documentation that requires almost no knowledge of Matrix inner workings to get a
+first basic setup running which relies on you reading the documentation in the right order:
+- [The Concepts](concepts.md) in few words.
+- [Getting Started](getting-started.md) step-by-step to a minimal working install.
+- [Identity stores](stores/README.md) you wish to fetch data from.
+- [Features](features) you are interested in that will use your Identity store(s) data.
+
+**IMPORTANT**: Be aware that mxisd tries to fit within the current protocol and existing products and basic understanding
+of the Matrix protocol is required for some advanced features.
+
+If all fails, come over to [the project room](https://matrix.to/#/#mxisd:kamax.io) and we'll do our best to get you
+started and answer questions you might have.
+
+### Do I need to use mxisd if I run a Homeserver?
+No, but it is strongly recommended, even if you don't use any Identity store or integration.
+
+In its default configuration, mxisd uses other federated public servers when performing queries.  
+It can also [be configured](features/identity.md#lookups) to use the central matrix.org servers, giving you access to at
+least the same information as if you were not running it.
+
+It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
+privacy consequences, which is not the case with the central Matrix.org servers.
+
+So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
+simple features on top of it.
+
+### I'm not sure I understand what an "Identity server" is supposed to be or do...
+The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
+what they want to do with that part of the ecosystem. Therefore, "Identity" is currently a misleading word and concept.
+Given the scope of the current Identity Service API, it would be best called "Invitation service".
+
+Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
+for groups/corporations/organisation. This is where mxisd comes in.
+
+mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
+up to its "Identity server" name.
+
+### Can I migrate my existing account on another Matrix server with mxisd?
+No.
+
+Accounts cannot currently migrate/move from one server to another.  
+See a [brief explanation document](concepts.md) about Matrix and mxisd concepts and vocabulary.
+
+### I already use the synapse LDAP3 auth provider. Why should I care about mxisd?
+The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) is not longer maintained and
+only handles on specific flow: validate credentials at login.
+
+It does not:
+- Auto-provision user profiles
+- Integrate with Identity management
+- Integrate with Directory searches
+
+mxisd is a replacement and enhancement of it, offering coherent results in all areas, which the LDAP3 auth provider
+does not.
+
+### Sydent is the official Identity server implementation of the Matrix team. Why not use that?
+You can, but [sydent](https://github.com/matrix-org/sydent):
+- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
+- is not meant to be linked to a specific Homeserver / domain
+- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
+- forces you to duplicate all your identity data, so people can be found by 3PIDs
+- forces users to enter all their emails and phone numbers manually in their profile
+
+So really, you should go with mxisd.
+
+### Will I loose access to the central Matrix.org/Vector.im Identity data if I use mxisd?
+No.
+
+In its default configuration, mxisd does not talk to the central Identity server matrix.org to avoid leaking your private
+data and those of people you might know.
+
+mxisd [can be configured](features/identity.md#lookups) to talk to the central Identity servers if you wish.
+
+### So mxisd is just a big hack! I don't want to use non-official features!
+mxisd primary concerns are your privacy and to always be compatible with the Matrix ecosystem and the Identity service API.  
+Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
+
+### Should I use mxisd if I don't host my own Homeserver?
+No.
+
+It is possible, but it is not supported and the scope of features will be extremely limited.
+Please consider hosting your own Homeserver and using mxisd alongside it.

docs/features/authentication.md

@@ -0,0 +1,203 @@
+# Authentication
+- [Description](#description)
+- [Basic](#basic)
+  - [Overview](#overview)
+  - [synapse](#synapse)
+  - [mxisd](#mxisd)
+  - [Validate](#validate)
+  - [Next steps](#next-steps)
+    - [Profile auto-fil](#profile-auto-fill)
+- [Advanced](#advanced)
+  - [Overview](#overview-1)
+  - [Requirements](#requirements)
+  - [Configuration](#configuration)
+    - [Reverse Proxy](#reverse-proxy)
+      - [Apache2](#apache2)
+    - [DNS Overwrite](#dns-overwrite)
+
+## Description
+Authentication is an enhanced feature of mxisd to ensure coherent and centralized identity management.  
+It allows to use Identity stores configured in mxisd to authenticate users on your Homeserver.
+
+Authentication is divided into two parts:
+- [Basic](#basic): authenticate with a regular username.
+- [Advanced](#advanced): same as basic with extra ability to authenticate using a 3PID.
+
+## Basic
+Authentication by username is possible by linking synapse and mxisd together using a specific module for synapse, also
+known as password provider.
+
+### Overview
+An overview of the Basic Authentication process:
+```
+                                                                                    Identity stores
+ Client                                                                             +------+
+   |                                            +-------------------------+    +--> | LDAP |
+   |   +---------------+  /_matrix/identity     | mxisd                   |    |    +------+
+   +-> | Reverse proxy | >------------------+   |                         |    |
+       +--|------------+                    |   |                         |    |    +--------+
+          |                                 +-----> Check ID stores     >------+--> | SQL DB |
+     Login request                          |   |                         |    |    +--------+
+          |                                 |   |     |                   |    |
+          |   +--------------------------+  |   +-----|-------------------+    +-->  ...
+          +-> | Homeserver               |  |         |
+              |                          |  |         |
+              | - Validate credentials >----+         |
+              |   Using REST auth module |            |
+              |                          |            |
+              | - Auto-provision <-------------------<+
+              |   user profiles          |    If valid credentials and supported by Identity store(s)
+              +--------------------------+
+```
+Performed on [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)
+
+### Synapse
+- Install the [password provider](https://github.com/kamax-io/matrix-synapse-rest-auth)
+- Edit your **synapse** configuration:
+  - As described by the auth module documentation
+  - Set `endpoint` to `http://mxisdAddress:8090` - Replace `mxisdAddress` by an IP/host name that provides a direct
+  connection to mxisd.  
+  This **MUST NOT** be a public address, and SHOULD NOT go through a reverse proxy.
+- Restart synapse
+
+### mxisd
+- Configure and enable at least one [Identity store](../stores/README.md)
+- Restart mxisd
+
+### Validate
+Login on the Homeserver using credentials present in one of your Identity stores.
+
+## Next steps
+### Profile auto-fill
+Auto-filling user profile depends on its support by your configured Identity stores.  
+See your Identity store [documentation](../stores/README.md) on how to enable the feature.
+
+
+## Advanced
+The Authentication feature allows users to:
+- Rewrite usernames matching a pattern to be mapped to another username via a 3PID.
+- login to their Homeserver by using their 3PIDs in a configured Identity store.
+
+This feature also allows to work around the following issues:
+- Lowercase all usernames for synapse, allowing case-insensitive login
+- Unable to login on synapse if username is numerical
+- Any generic transformation of username prior to sending to synapse, bypassing the restriction that password providers
+cannot change the localpart being authenticated.
+
+### Overview
+This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/login` as depicted below:
+```
+            +----------------------------+
+            |  Reverse Proxy             |
+            |                            |
+            |                            |     Step 1    +---------------------------+     Step 2
+            |                            |               |                           |
+Client+---->| /_matrix/client/r0/login +---------------->|                           | Look up address  +---------+
+            |                      ^     |               |  mxisd - Identity server  +----------------->| Backend |
+            |                      |     |               |                           |                  +---------+
+            | /_matrix/* +--+      +---------------------+                           |
+            |               |            |               +---------------+-----------+
+            |               |            |     Step 4                    |
+            |               |            |                               | Step 3
+            +---------------|------------+                               |
+                            |                                            | /_matrix/client/r0/login
+                            |                       +--------------+     |
+                            |                       |              |     |
+                            +---------------------->|  Homeserver  |<----+
+                                                    |              |
+                                                    +--------------+
+
+```
+
+Steps of user authentication using a 3PID:
+1. The intercepted login request is directly sent to mxisd instead of the Homeserver.
+2. Identity stores are queried for a matching user identity in order to modify the request to use the user name.
+3. The Homeserver, from which the request was intercepted, is queried using the request at previous step.
+   Its address is resolved using the DNS Overwrite feature to reach its internal address on a non-encrypted port.
+4. The response from the Homeserver is sent back to the client, believing it was the HS which directly answered.
+
+### Requirements
+- Compatible [Identity store](../stores/README.md)
+- [Basic Authentication configured and working](#basic)
+- Client and Homeserver using the [C2S API r0.4.x](https://matrix.org/docs/spec/client_server/r0.4.0.html) or later
+- Reverse proxy setup
+
+### Configuration
+#### Reverse Proxy
+##### Apache2
+The specific configuration to put under the relevant `VirtualHost`:
+```apache
+ProxyPass /_matrix/client/r0/login http://localhost:8090/_matrix/client/r0/login
+```
+`ProxyPreserveHost` or equivalent **must** be enabled to detect to which Homeserver mxisd should talk to when building results.
+
+Your VirtualHost should now look similar to:
+```apache
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/client/r0/login http://localhost:8090/_matrix/client/r0/login
+    ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
+    ProxyPass /_matrix http://localhost:8008/_matrix
+</VirtualHost>
+```
+
+#### DNS Overwrite
+Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
+the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
+
+To do so, put the following configuration in your mxisd configuration:
+```yaml
+dns:
+  overwrite:
+    homeserver:
+      client:
+        - name: 'example.org'
+          value: 'http://localhost:8008'
+```
+`name` must be the hostname of the URL that clients use when connecting to the Homeserver.
+You can use `${server.name}` to auto-populate the `value` using the `server.name` configuration option and avoid duplicating it.
+In case the hostname is the same as your Matrix domain and `server.name` is not explicitely set in the config, `server.name` will default to
+`matrix.domain` and will still probably have the correct value.
+
+`value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
+
+#### Username rewrite
+In mxisd config:
+```yaml
+auth:
+  rewrite:
+    user:
+      rules:
+        - regex: <your regexp>
+          medium: 'your.custom.medium.type'
+```
+`rules` takes a list of rules. Rules have two properties:
+- `regexp`: The regex pattern to match. This **MUST** match the full string. See [Java regex](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html) for syntax.
+- `medium`: Custom 3PID type that will be used in the 3PID lookup. This can be anything you want and needs to be supported
+by your Identity store config and/or code.
+
+Rules are matched in listed order.
+
+Common regexp patterns:
+- Numerical usernames: `[0-9]+`
+
+##### LDAP Example
+If your users use their numerical employee IDs, which cannot be used with synapse, you can make it work with (relevant config only):
+```yaml
+auth:
+  rewrite:
+    user:
+      rules:
+        - regex: '[0-9]+'
+          medium: 'kmx.employee.id'
+          
+ldap:
+  attribute:
+    threepid:
+      kmx.employee.id:
+        - 'ldapAttributeForEmployeeId'
+```

docs/features/bridge-integration.md

@@ -0,0 +1,31 @@
+# Bridge Integration
+To help natural bridge integration into the regular usage of a Matrix client, mxisd provides a way for bridge to reply
+to 3PID queries if no mapping was found, allowing seamless bridging to a network.
+
+This is performed by implementing a specific endpoint on the bridge to map a 3PID lookup to a virtual user.
+
+**NOTE**: This document is incomplete and might be misleading. In doubt, come in our Matrix room.  
+You can also look at our [Email Bridge README](https://github.com/kamax-matrix/matrix-appservice-email#mxisd) for an example
+of working configuration.
+
+## Configuration
+```yaml
+lookup:
+  recursive:
+    bridge:
+      enabled: <boolean>
+      recursiveOnly: <boolean>
+      server: <URL to the bridge endpoint for all 3PID medium>
+      mappings:
+        <3PID MEDIUM HERE>: <URL to dedicated bridge for that medium>
+
+```
+
+## Integration
+Implement a simplified version of the [Identity service single lookup endpoint](https://kamax.io/matrix/api/identity_service/unstable.html#get-matrix-identity-api-v1-lookup)
+with only the following parameters needed:
+- `address`
+- `medium`
+- `mxid`
+
+Or an empty object if no resolution exists or desired.

docs/features/directory.md

@@ -0,0 +1,153 @@
+# User Directory
+- [Description](#description)
+- [Overview](#overview)
+- [Requirements](#requirements)
+- [Configuration](#configuration)
+  - [Reverse Proxy](#reverse-proxy)
+    - [Apache2](#apache2)
+    - [nginx](#nginx)
+  - [DNS Overwrite](#dns-overwrite)
+- [Next steps](#next-steps)
+
+## Description
+This feature allows you to search for existing and/or potential users that are already present in your Identity backend
+or that already share a room with you on the Homeserver.
+
+Without any integration, synapse:
+- Only search within the users **already** known to you or in public rooms
+- Only search on the Display Name and the Matrix ID
+
+By enabling this feature, you can by default:
+- Search on Matrix ID, Display name and 3PIDs (Email, phone numbers) of any users already in your configured backend
+- Search for users which you are not in contact with yet. Super useful for corporations who want to give Matrix access
+internally, so users can just find themselves **prior** to having any common room(s)
+- Add extra attributes of your backend to extend the search
+- Include your homeserver search results to those found by mxisd
+
+By integrating mxisd, you get the default behaviour and a bunch of extras, ensuring your users will always find each other.
+
+## Overview
+This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/user_directory/search` like so:
+```
+           +----------------------------------------------+
+Client --> | Reverse proxy                                                                         Step 2
+           |                                              Step 1    +-------------------------+
+           |   /_matrix/client/r0/user_directory/search ----------> |                         |  Search in   +---------+
+           |                        /\                              | mxisd - Identity server | -----------> | Backend |
+           |   /_matrix/*            \----------------------------- |                         |  all users   +---------+
+           |        |            Step 4: Send back merged results   +-------------------------+
+           +        |                                                            |
+                    |                                                          Step 3
+                    |                                                            |
+                    |    +------------+                                Search in known users
+                    \--> | Homeserver | <----------------------------------------/
+                         +------------+   /_matrix/client/r0/user_directory/search
+```
+Steps:
+1. The intercepted request is directly sent to mxisd instead of the Homeserver.
+2. Identity stores are queried for any match on the search value sent by the client.
+3. The Homeserver, from which the request was intercepted, is queried using the same request as the client.
+   Its address is resolved using the DNS Overwrite feature to reach its internal address on a non-encrypted port.
+4. Results from Identity stores and the Homeserver are merged together and sent back to the client, believing it was the HS
+which directly answered the request.
+
+## Requirements
+- Reverse proxy setup, which you should already have in place if you use mxisd
+- At least one compatible [Identity store](../stores/README.md) enabled
+  
+## Configuration
+### Reverse Proxy
+#### Apache2
+The specific configuration to put under the relevant `VirtualHost`:
+```apache
+ProxyPass /_matrix/client/r0/user_directory/ http://0.0.0.0:8090/_matrix/client/r0/user_directory/
+```
+`ProxyPreserveHost` or equivalent must be enabled to detect to which Homeserver mxisd should talk to when building
+results.
+
+Your `VirtualHost` should now look like this:
+```apache
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/client/r0/user_directory/ http://localhost:8090/_matrix/client/r0/user_directory/
+    ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
+    ProxyPass /_matrix http://localhost:8008/_matrix
+</VirtualHost>
+```
+
+#### nginx
+The specific configuration to add under your `server` section is:
+```nginx
+location /_matrix/client/r0/user_directory {
+    proxy_pass http://0.0.0.0:8090/_matrix/client/r0/user_directory;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $remote_addr;
+}
+```
+
+Your `server` section should now look like this:
+```nginx
+server {
+    listen 443 ssl;
+    server_name example.org;
+    
+    ...
+    
+    location /_matrix/client/r0/user_directory {
+        proxy_pass http://localhost:8090/_matrix/client/r0/user_directory;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
+### DNS Overwrite
+Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
+the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
+
+To do so, use the following configuration:
+```yaml
+dns:
+  overwrite:
+    homeserver:
+      client:
+        - name: 'example.org'
+          value: 'http://localhost:8008'
+```
+- `name` must be the hostname of the URL that clients use when connecting to the Homeserver.
+- `value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
+
+## Next steps
+### Homeserver results
+You can configure if the Homeserver should be queried at all when doing a directory search.  
+To disable Homeserver results, set the following in mxisd configuration file:
+```yaml
+directory:
+  exclude:
+    homeserver: true
+```
+
+### 3PID exclusion in search
+You can configure if the 3PID should also be included when doing a directory search.
+By default, a search is performed on the 3PIDs. If you would like to not include them:
+```yaml
+directory:
+  exclude:
+    threepid: true
+```

docs/features/experimental/application-service.md

@@ -0,0 +1,72 @@
+# Integration as an Application Service
+**WARNING:** These features are currently highly experimental. They can be removed or modified without notice.  
+All the features requires a Homeserver capable of connecting Application Services.
+
+## Email notification for Room invites by Matrix ID
+This feature allows for users found in Identity stores to be instantly notified about Room Invites, regardless if their
+account was already provisioned on the Homeserver.
+
+### Requirements
+- [Identity store(s)](../../stores/README.md) supporting the Profile feature
+- At least one email entry in the identity store for each user that could be invited.
+
+### Configuration
+In your mxisd config file:
+```yaml
+matrix:
+  listener:
+    url:  '<URL TO THE CS API OF THE HOMESERVER>'
+    localpart: 'appservice-mxisd'
+    token:
+      hs: 'HS_TOKEN_CHANGE_ME'
+
+synapseSql:
+  enabled: false ## Do not use this line if Synapse is used as an Identity Store
+  type: '<DB TYPE>'
+  connection: '<DB CONNECTION URL>'
+```
+
+The `synapseSql` section is used to retrieve display names which are not directly accessible in this mode.
+For details about `type` and `connection`, see the [relevant documentation](../../stores/synapse.md).
+If you do not configure it, some placeholders will not be available in the notification, like the Room name.
+
+You can also change the default template of the notification using the `generic.matrixId` template option.  
+See [the Template generator documentation](../../threepids/notification/template-generator.md) for more info.
+
+### Homeserver integration
+#### Synapse
+Create a new appservice registration file. Futher config will assume it is in `/etc/matrix-synapse/appservice-mxisd.yaml`
+```yaml
+id: "appservice-mxisd"
+url: "http://127.0.0.1:8090"
+as_token: "AS_TOKEN_CHANGE_ME"
+hs_token: "HS_TOKEN_CHANGE_ME"
+sender_localpart: "appservice-mxisd"
+namespaces:
+  users:
+    - regex: "@*"
+      exclusive: false
+  aliases: []
+  rooms: []
+```
+`id`: An arbitrary unique string to identify the AS.  
+`url`: mxisd to reach mxisd. This ideally should be HTTP and not going through any reverse proxy.  
+`as_token`: Arbitrary value used by mxisd when talking to the HS. Not currently used.  
+`hs_token`: Arbitrary value used by synapse when talking to mxisd. Must match `token.hs` in mxisd config.
+`sender_localpart`: Username for the mxisd itself on the HS. Default configuration should be kept.  
+`namespaces`: To be kept as is.  
+
+Edit your `homeserver.yaml` and add a new entry to the appservice config file, which should look something like this:
+```yaml
+app_service_config_files:
+  - '/etc/matrix-synapse/appservice-mxisd.yaml'
+  - ...
+```
+
+Restart synapse when done to register mxisd.
+
+#### Others
+See your Homeserver documentation on how to integrate.
+
+### Test
+Invite a user which is part of your domain while an appropriate Identity store is used.

docs/features/experimental/profile.md

@@ -0,0 +1,16 @@
+# Profile
+**WARNING**: The following sub-features are considered experimental and not officially supported. Use at your own peril.
+
+## Public Profile enhancement
+This feature allows to enhance a public profile query with more info than just Matrix ID and Display name, allowing for
+custom applications to retrieve custom data not currently provided by synapse, per example.
+
+**WARNING**: This information can be queried without authentication as per the specification. Do not enable unless in a
+controlled environment.
+
+### Configuration
+#### Reverse proxy
+##### Apache
+```apache
+ProxyPassMatch "^/_matrix/client/r0/profile/([^/]+)$" "http://127.0.0.1:8090/_matrix/client/r0/profile/$1"
+```

docs/features/federation.md

@@ -0,0 +1,60 @@
+# Federation
+Federation is the process by which domain owners can make compatible 3PIDs mapping auto-discoverable by looking for another
+Federated Identity server using the DNS domain part of the 3PID.
+
+Emails are the best candidate for this kind of resolution which are DNS domain based already.  
+On the other hand, Phone numbers cannot be resolved this way.
+
+For 3PIDs which are not compatible with the DNS system, mxisd can be configured to talk to fallback Identity servers like
+the central matrix.org one. See the [Identity feature](identity.md#lookups) for instructions on how to enable it.
+
+Outbound federation is enabled by default while inbound federation is opt-in and require a specific DNS record.
+
+## Overview
+```
+              +-------------------+   +-------------> +----------+
+              | mxisd             |   |               | Backends |
+              |                   |   |      +------> +----------+
+              |                   |   |      |
+              | Invites / Lookups |   |      |
+ Federated    | +--------+        |   |      |
+ Identity  ---->| Remote |>-----------+      |
+ Server       | +--------+        |          |
+              |                   |          |
+              | +--------+        |          |        +-------------------+
+ Homeserver --->| Local  |>------------------+------> | Remote Federated  |
+ and clients  | +--------+        |                   | mxisd servers     |
+              +-------------------+                   +-------------------+
+```
+
+## Inbound
+If you would like to be reachable for lookups over federation, create the following DNS SRV entry and replace
+`matrix.example.com` by your Identity server public hostname:
+```
+_matrix-identity._tcp.example.com. 3600 IN SRV 10 0 443 matrix.example.com.
+``` 
+
+The port must be HTTPS capable which is what you get in a regular setup with a reverse proxy from 443 to TCP 8090 of mxisd.
+
+## Outbound
+If you would like to disable outbound federation and isolate your identity server from the rest of the Matrix network,
+use the following mxisd configuration options:
+```yaml
+lookup:
+  recursive:
+    enabled: false
+invite:
+  resolution:
+    recursive: false
+session:
+  policy:
+    validation:
+      forLocal:
+        toRemote:
+          enabled: false
+      forRemote:
+        toRemote:
+          enabled: false
+``` 
+
+There is currently no way to selectively disable federation towards specific servers, but this feature is planned.

docs/features/identity.md

@@ -0,0 +1,32 @@
+# Identity
+**WARNING**: This document is incomplete and can be misleading.
+
+Implementation of the [Unofficial Matrix Identity Service API](https://kamax.io/matrix/api/identity_service/unstable.html).
+
+## Lookups
+If you would like to use the central matrix.org Identity server to ensure maximum discovery at the cost of potentially
+leaking all your contacts information, add the following to your configuration:
+```yaml
+forward:
+  servers:
+    - 'matrix-org'
+```
+**NOTE:** You should carefully consider enabling this option, which is discouraged.  
+For more info, see the [relevant issue](https://github.com/kamax-matrix/mxisd/issues/76).
+
+## Room Invitations
+Resolution can be customized using the following configuration:
+
+`invite.resolution.recursive`  
+- Default value: `true`  
+- Description: Control if the pending invite resolution should be done recursively or not.  
+  **DANGER ZONE:** This setting has the potential to create "an isolated island", which can have unexpected side effects
+  and break invites in rooms. This will most likely not have the effect you think it does. Only change the value if you
+  understand the consequences.
+
+`invite.resolution.timer`  
+- Default value: `1`  
+- Description: How often, in minutes, mxisd should try to resolve pending invites.
+
+## 3PID addition to user profile
+See the [3PID session documents](../threepids/session)

docs/features/profile.md

@@ -0,0 +1,10 @@
+# Profile
+The profile feature does not do anything on its own and acts as a support feature for others, allowing to retrieve
+information about a user based on its Matrix ID by querying enabled [Identity stores](../stores/README.md).
+
+Currently supported:
+- Display name
+- 3PIDs
+- Roles/Groups
+
+Experimental sub-features are also available. See [the dedicated document](experimental/profile.md).

docs/getting-started.md

@@ -0,0 +1,158 @@
+# Getting started
+1. [Preparation](#preparation)
+2. [Install](#install)
+3. [Configure](#configure)
+4. [Integrate](#integrate)
+5. [Validate](#validate)
+6. [Next steps](#next-steps)
+
+Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
+talk to the central Matrix.org Identity server.  
+This will be a good ground work for further integration with features and your existing Identity stores.
+
+---
+
+If you would like a more fully integrated setup out of the box, the [matrix-docker-ansible-deploy](https://github.com/spantaleev/matrix-docker-ansible-deploy)
+project provides a turn-key full-stack solution, including LDAP and the various mxisd features enabled and ready.  
+We work closely with the project owner so the latest mxisd version is always supported.
+
+If you choose to use it, this Getting Started guide is not applicable - See the project documentation. You may then
+directly go to the [Next steps](#next-steps).
+
+## Preparation
+You will need:
+- Working Homeserver, ideally with working federation
+- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
+
+As synapse requires an HTTPS connection when talking to an Identity service, **a reverse proxy is required** as mxisd does
+not support HTTPS listener at this time.
+
+For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.
+
+Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-matrix/mxisd/wiki/Gotchas#nating) if you use the same
+hostname.
+
+The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
+If you would like a high-level view of the infrastructure and how each feature is integrated, see the
+[dedicated document](architecture.md)
+
+## Install
+Install via:
+- [Docker image](install/docker.md)
+- [Debian package](install/debian.md)
+- [ArchLinux](install/archlinux.md)
+- [NixOS](install/nixos.md)
+- [Sources](build.md)
+
+See the [Latest release](https://github.com/kamax-matrix/mxisd/releases/latest) for links to each.
+
+## Configure
+> **NOTE**: Please view the install instruction for your platform, as this step might be optional or already handled for you.
+  
+> **NOTE**: Details about configuration syntax and format are described [here](configure.md)
+
+Create/edit a minimal configuration (see installer doc for the location):
+```yaml
+matrix:
+  domain: 'example.org'
+key:
+  path: '/path/to/signing.key.file'
+storage:
+  provider:
+    sqlite:
+      database: '/path/to/mxisd.db'
+```  
+- `matrix.domain` should be set to your Homeserver domain (`server_name` in synapse configuration)
+- `key.path` will store the signing keys, which must be kept safe! If the file does not exist, keys will be generated for you.
+- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
+
+If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
+Complete configuration guide is available [here](configure.md).
+
+## Integrate
+For an overview of a typical mxisd infrastructure, see the [dedicated document](architecture.md)
+### Reverse proxy
+#### Apache2
+In the `VirtualHost` section handling the domain with SSL, add the following and replace `0.0.0.0` by the internal
+hostname/IP pointing to mxisd.  
+**This line MUST be present before the one for the homeserver!**
+```apache
+ProxyPass /_matrix/identity http://0.0.0.0:8090/_matrix/identity
+```
+
+Typical configuration would look like:
+```apache
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
+    ProxyPass /_matrix http://localhost:8008/_matrix
+</VirtualHost>
+```
+
+#### nginx
+In the `server` section handling the domain with SSL, add the following and replace `0.0.0.0` with the internal
+hostname/IP pointing to mxisd.
+**This line MUST be present before the one for the homeserver!**
+```nginx
+location /_matrix/identity {
+    proxy_pass http://0.0.0.0:8090/_matrix/identity;
+}
+```
+
+Typical configuration would look like:
+```nginx
+server {
+    listen 443 ssl;
+    server_name example.org;
+    
+    ...
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
+### Synapse
+Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
+In a typical configuration, you would end up with something similar to:
+```yaml
+trusted_third_party_id_servers:
+    - example.org
+```
+It is recommended to remove `matrix.org` and `vector.im` (or any other default entry) from your configuration so only
+your own Identity server is authoritative for your HS.
+
+## Validate
+**NOTE:** In case your homeserver has no working federation, step 5 will not happen. If step 4 took place, consider
+your installation validated.
+
+1. Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
+the relevant hostname which you configured in your reverse proxy.
+2. Create a new empty room. All further actions will take place in this room.
+3. Invite `mxisd-federation-test@kamax.io`
+4. The 3PID invite should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.
+5. The invited test user will join the room, send a congratulation message and leave.
+**NOTE:** You might not see a suggestion for the e-mail address, which is normal. Still proceed with the invite.
+  
+If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
+If it did not work, [get in touch](../README.md#support) and we'll do our best to get you started.
+
+## Next steps
+Once your mxisd server is up and running, there are several ways you can enhance and integrate further with your
+infrastructure:
+
+- [Enable extra features](features/)
+- [Use your own Identity stores](stores/README.md)

docs/install/archlinux.md

@@ -0,0 +1,3 @@
+# Arch Linux package
+An Arch Linux package in the AUR repos is maintained by [r3pek](https://matrix.to/#/@r3pek:r3pek.org), a community member.  
+See https://aur.archlinux.org/packages/mxisd/

docs/install/debian.md

@@ -0,0 +1,42 @@
+# Debian package
+## Requirements
+- Any distribution that supports Java 8
+
+## Install
+1. Download the [latest release](https://github.com/kamax-matrix/mxisd/releases/latest)
+2. Run:
+```bash
+dpkg -i /path/to/downloaded/mxisd.deb
+```
+## Files
+| Location                            | Purpose                                      |
+|-------------------------------------|----------------------------------------------|
+| `/etc/mxisd`                        | Configuration directory                      |
+| `/etc/mxisd/mxisd.yaml`             | Main configuration file                      |
+| `/etc/systemd/system/mxisd.service` | Systemd configuration file for mxisd service |
+| `/usr/lib/mxisd`                    | Binaries                                     |
+| `/var/lib/mxisd`                    | Data                                         |
+| `/var/lib/mxisd/signing.key`        | Default location for mxisd signing keys      |
+
+## Control
+Start mxisd using:
+```bash
+sudo systemctl start mxisd
+```
+
+Stop mxisd using:
+```bash
+sudo systemctl stop mxisd
+```
+
+## Troubleshoot
+All logs are sent to `STDOUT` which are saved in `/var/log/syslog` by default.  
+You can:
+- grep & tail using `mxisd`:
+```
+tail -n 99 -f /var/log/syslog | grep mxisd
+```
+- use Systemd's journal:
+```
+journalctl -f -n 99 -u mxisd
+```

docs/install/docker.md

@@ -0,0 +1,22 @@
+# Docker
+## Fetch
+Pull the latest stable image:
+```bash
+docker pull kamax/mxisd
+```
+
+## Configure
+On first run, simply using `MATRIX_DOMAIN` as an environment variable will create a default config for you.  
+You can also provide a configuration file named `mxisd.yaml` in the volume mapped to `/etc/mxisd` before starting your
+container.
+
+## Run
+Use the following command after adapting to your needs:
+- The `MATRIX_DOMAIN` environment variable to yours
+- The volumes host paths
+
+```bash
+docker run --rm -e MATRIX_DOMAIN=example.org -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
+```
+
+For more info, including the list of possible tags, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)

docs/install/nixos.md

@@ -0,0 +1,8 @@
+# NixOS package
+mxisd is available as a NixOS package in the official repos.
+
+It is maintained by [maximilian](https://matrix.to/#/@maximilian:transformierende-gesellschaft.org), a community member.  
+
+Related resources:
+- [NixOS](https://nixos.org/)
+- [The module definition](https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/networking/mxisd.nix)

docs/install/source.md

@@ -0,0 +1,44 @@
+# Install from sources
+## Instructions
+Follow the [build instructions](../build.md) then:
+
+### Prepare files and directories:
+```bash
+# Create a dedicated user
+useradd -r mxisd
+
+# Create config directory and set ownership
+mkdir -p /etc/mxisd
+
+# Create data directory and set ownership
+mkdir -p /var/lib/mxisd
+chown -R mxisd /var/lib/mxisd
+
+# Create bin directory, copy the jar and launch scriot to bin directory
+mkdir /usr/lib/mxisd
+cp ./build/libs/mxisd.jar /usr/lib/mxisd/
+cp ./src/script/mxisd /usr/lib/mxisd
+chown -R mxisd /usr/lib/mxisd
+chmod a+x /usr/lib/mxisd/mxisd
+
+# Create symlink for easy exec
+ln -s /usr/lib/mxisd/mxisd /usr/bin/mxisd
+```
+
+### Prepare config file
+Copy the sample config file `./mxisd.example.yaml` to `/etc/mxisd/mxisd.yaml`, edit to your needs
+
+### Prepare Systemd
+1. Copy `src/systemd/mxisd.service` to `/etc/systemd/system/` and edit if needed
+2. Enable service for auto-startup
+```bash
+systemctl enable mxisd
+```
+
+### Run
+```bash
+systemctl start mxisd
+```
+
+## Debug
+mxisd logs to stdout, which is normally sent to `/var/log/syslog` or `/var/log/messages`.

docs/stores/README.md

@@ -0,0 +1,8 @@
+# Identity Stores
+- [Synapse](synapse.md) - Turn your SynapseDB into a self-contained Identity store
+- [LDAP-based](ldap.md) - Any LDAP-based product like Active Directory, Samba, NetIQ, OpenLDAP
+- [SQL Databases](sql.md) - Most common databases like MariaDB, MySQL, PostgreSQL, SQLite
+- [Website / Web service / Web app](rest.md) - Arbitrary REST endpoints
+- [Executables](exec.md) - Run arbitrary executables with configurable stdin, arguments, environment and stdout
+- [Wordpress](wordpress.md) - Connect your Wordpress-powered website DB
+- [Google Firebase](firebase.md) - Use your Firebase users (with experimental SSO support!)

docs/stores/exec.md

@@ -0,0 +1,498 @@
+# Exec Identity Store
+- [Features](#features)
+- [Overview](#overview)
+- [Configuration](#configuration)
+  - [Global](#global)
+    - [Tokens](#tokens)
+  - [Executable](#executable)
+    - [Input](#input)
+    - [Output](#output)
+  - [Examples](#examples)
+  - [Per-Feature](#per-feature)
+- [Authentication](#authentication)
+  - [Tokens](#tokens-1)
+  - [Input](#input-1)
+  - [Output](#output-1)
+- [Directory](#directory)
+  - [Tokens](#tokens-2)
+  - [Input](#input-2)
+  - [Output](#output-2)
+- [Identity](#identity)
+  - [Single Lookup](#single-lookup)
+    - [Tokens](#tokens-3)
+    - [Input](#input-3)
+    - [Output](#output-3)
+  - [Bulk Lookup](#bulk-lookup)
+    - [Tokens](#tokens-4)
+    - [Input](#input-4)
+    - [Output](#output-4)
+- [Profile](#profile)
+  - [Tokens](#tokens-5)
+  - [Input](#input-5)
+  - [Output](#output-5)
+  
+---
+
+## Features
+|                       Name                      | Supported |
+|-------------------------------------------------|-----------|
+| [Authentication](../features/authentication.md) | Yes       |
+| [Directory](../features/directory.md)           | Yes       |
+| [Identity](../features/identity.md)             | Yes       |
+| [Profile](#profile)                             | Yes       |
+
+This Identity Store lets you run arbitrary commands to handle the various requests in each support feature.  
+It is the most versatile Identity store of mxisd, allowing you to connect any kind of logic with any executable/script.
+
+## Overview
+Each request can be mapping to a fully customizable command configuration.  
+The various parameters can be provided via any combination of:
+- [Standard Input](https://en.wikipedia.org/wiki/Standard_streams#Standard_input_(stdin))
+- [Command-line arguments](https://en.wikipedia.org/wiki/Command-line_interface#Arguments)
+- [Environment variables](https://en.wikipedia.org/wiki/Environment_variable)
+
+Each of those supports a set of customizable token which will be replaced prior to running the command, allowing to
+provide the input values in any number of ways.
+
+Success and data will be provided via any combination of:
+- [Exit status](https://en.wikipedia.org/wiki/Exit_status)
+- [Standard Output](https://en.wikipedia.org/wiki/Standard_streams#Standard_output_(stdout))
+
+Each of those supports a set of configuration item to decide how to process the value and/or in which format.
+
+All values, inputs and outputs are UTF-8 encoded.
+
+## Configuration
+Each feature comes with a set of possible lookup/action which is mapped to a generic configuration item block.  
+We will use the term `Executable` for each lookup/action and `Processor` for each configuration block.
+
+### Global
+```yaml
+exec:
+  enabled: <boolean>
+```
+Enable/disable the Identity store at a global/default level. Each feature can still be individually enabled/disabled.
+
+#### Tokens
+The following options allow to globally set tokens for value replacement across all features and processors config.  
+Not all features use all tokens, and each feature might also have its own specific tokens. See each feature documentation.
+  
+They can be set within the following scope:
+
+```yaml
+exec:
+  token:
+    <token>: '<value>'
+```
+
+---
+
+The following tokens and default values are available:
+```yaml
+localpart: '{localpart}'
+```
+Localpart of Matrix User IDs
+
+```yaml
+domain: '{domain}'
+```
+Domain of Matrix User IDs
+
+```yaml
+mxid: '{mxid}'
+```
+Full representation of Matrix User IDs
+
+```yaml
+medium: '{medium}'
+```
+Medium of 3PIDs
+
+```yaml
+address: '{address}'
+```
+Address of 3PIDs
+
+```yaml
+type: '{type}'
+```
+Type of query
+
+```yaml
+query: '{query}'
+```
+Query value
+
+### Executable
+*Executable*s have the following options:
+```yaml
+command: '/path/to/executableOrScript'
+
+```
+Set the executable (relative or absolute) path to be executed. If no command is given, the action will return a "neutral"
+result if possible or be skipped altogether.
+
+---
+
+Command line arguments can be given via a list via both YAML formats:
+```yaml
+args:
+ - '-t'
+ - '{token}'
+ - '-v'
+ - 'value'
+```
+or
+```yaml
+args: ['-t', '{token}', '-v', 'value]
+```
+Each argument will be processed for token replacement.
+
+---
+
+Environment variables can be given as key/value pairs:
+```yaml
+env:
+  ENV_VAR_1: 'value'
+  ENV_VAR_2: '{token}'
+``` 
+Each variable value will be processed for token replacement.
+
+#### Input
+Standard input can be configured in the namespaces `input` with:
+- `type`: The format to use
+- `template`: The full or partial template with tokens to be used when generating the input
+
+Not all features and *Executable*s allow for a template to be provided.  
+Templates for listed-based input are not supported at this time.  
+Default templates may be provided per *Executable*.
+
+The following types are available:
+- `json`: Use JSON format, shared with the [REST Identity Store](rest.md)
+- `plain`: Use a custom multi-lines, optionally tab-separated input
+
+#### Output
+Standard output can be configured in the namespaces `output` with:
+- `type`: The format to use
+- `template`: The full or partial template with tokens to be used when processing the output
+
+Not all features and *Executable*s allow for a template to be provided.  
+Templates for listed-based output are not supported at this time.  
+Default templates may be provided per *Executable*.
+
+The following types are available:
+- `json`: Use JSON format, shared with the [REST Identity Store](rest.md)
+- `plain`: Use a custom multi-lines, optionally tab-separated output
+
+### Examples
+#### Basic
+```yaml
+exec:
+  auth:
+    enabled: true
+    command: '/opt/mxisd-exec/auth.sh'
+    args: ['{localpart}']
+    input:
+      type: 'plain'
+      template: '{password}'
+  env:
+    DOMAIN: '{domain}'
+```
+With Authentication enabled, run `/opt/mxisd-exec/auth.sh` when validating credentials, providing:
+- A single command-line argument to provide the `localoart` as username 
+- A plain text string with the password token for standard input, which will be replaced by the password to check
+- A single environment variable `DOMAIN` containing Matrix ID domain, if given
+
+The command will use the default values for:
+- Success exit status of `0`
+- Failure exit status of `1`
+- Any other exit status considered as error
+- The standard output processing as not processed
+
+#### Advanced
+Given the fictional `placeholder` feature:
+```yaml
+exec.enabled: true
+exec.token.mxid: '{matrixId}'
+
+exec.placeholder.token.localpart: '{username}'
+exec.placeholder.command: '/path/to/executable'
+exec.placeholder.args:
+  - '-u'
+  - '{username}'
+exec.placeholder.env:
+  MATRIX_DOMAIN: '{domain}'
+  MATRIX_USER_ID: '{matrixId}'
+  
+exec.placeholder.output.type: 'json'
+exec.placeholder.exit.success: [0, 128]
+exec.placeholder.exit.failure: [1, 129]
+```
+With:
+- The Identity store enabled for all features
+- A global specific token `{matrixId}` for Matrix User IDs, replacing the default `{mxid}`
+
+Running `/path/to/executable` providing:
+- A custom token for localpart, `{username}`, used as a 2nd command-line argument
+- An extracted Matrix User ID `localpart` provided as the second command line argument, the first one being `-u` 
+- A password, the extracted Matrix `domain` and the full User ID as arbitrary environment variables, respectively
+  `PASSWORD`, `MATRIX_DOMAIN` and `MATRIX_USER_ID`
+
+After execution:
+- Process stdout as [JSON](https://en.wikipedia.org/wiki/JSON)
+- Consider exit status `0` and `128` as success and try to process the stdout for data
+- Consider exit status `1` and `129` as failure and try to process the stdout for error code and message
+
+### Per Feature
+See each dedicated [Feature](#features) section.
+
+## Authentication
+The Authentication feature can be enabled/disabled using:
+```yaml
+exec:
+  auth:
+    enabled: <true/false>
+```
+
+---
+
+This feature provides a single *Executable* under the namespace:
+```yaml
+exec:
+  auth:
+  ...
+```
+
+### Tokens
+The following tokens/default values are specific to this feature:
+```yaml
+password: '{password}'
+```
+The provided password
+
+### Input
+Supported input types and default templates:
+
+#### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+#### Plain (`plain`)
+Default template:
+```
+{localpart}
+{domain}
+{mxid}
+{password}
+```
+
+### Output
+Supported output types and default templates:
+
+#### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+#### Plain (`plain`)
+**NOTE:** This has limited support. Use the JSON type for full support.
+
+Default template:
+```
+[success status, true or 1 are interpreted as success]
+[display name of the user]
+```
+
+## Directory
+The Directory feature can be enabled/disabled using:
+```yaml
+exec:
+  directory:
+    enabled: <true/false>
+```
+
+---
+
+Two search types configuration namespace are available, using the same input/output formats and templates:
+
+By name:
+```yaml
+exec:
+  directory:
+    search:
+      byName:
+        ...
+```
+By 3PID:
+```yaml
+exec:
+  directory:
+    search:
+      byThreepid:
+        ...
+```
+
+#### Tokens
+No specific tokens are available.
+
+#### Input
+Supported input types and default templates:
+
+##### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+##### Plain (`plain`)
+Default template:
+```
+[type of search, following the REST Identity store format]
+[query string]
+```
+
+#### Output
+Supported output types and default templates:
+
+##### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+##### Plain (`plain`)
+**Not supported at this time.** Use the JSON type.
+
+## Identity
+The Identity feature can be enabled/disabled using:
+```yaml
+exec.identity.enabled: <true/false>
+```
+
+### Single lookup
+Configuration namespace:
+```yaml
+exec.identity.lookup.single:
+  ...
+```
+
+#### Tokens
+No specific tokens are available.
+
+#### Input
+Supported input types and default templates:
+
+##### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+##### Plain (`plain`)
+Default template:
+```
+{medium}
+{address}
+```
+
+#### Output
+Supported output types and default templates:
+
+##### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+##### Plain (`plain`)
+Default template:
+```
+[User ID type, as documented in the REST Identity Store]
+[User ID value]
+```
+
+The User ID type will default to `localpart` if:
+- Only one line is returned
+- The first line is empty
+
+### Bulk lookup
+Configuration namespace:
+```yaml
+exec:
+  identity:
+    lookup:
+      bulk:
+        ...
+```
+
+#### Tokens
+No specific tokens are available.
+
+#### Input
+Supported input types and default templates:
+
+##### JSON (`json`)
+**NOTE:** Custom Templates are not supported.
+
+Same as the [REST Identity Store](rest.md).
+
+##### Plain (`plain`)
+**Not supported at this time.** Use the JSON type.
+
+#### Output
+Supported output types and default templates:
+
+##### JSON (`json`)
+**NOTE:** Custom Templates are not supported.
+
+Same as the [REST Identity Store](rest.md).
+
+##### Plain (`plain`)
+**Not supported at this time.** Use the JSON type.
+
+## Profile
+The Profile feature can be enabled/disabled using:
+```yaml
+exec:
+  profile:
+    enabled: <true/false>
+```
+
+---
+
+The following *Executable*s namespace are available, share the same input/output formats and templates:
+
+Get Display name:
+```yaml
+exec:
+  profile:
+    displayName:
+      ...
+```
+
+Get 3PIDs:
+```yaml
+exec:
+  profile:
+    threePid:
+      ...
+```
+
+Get Roles:
+```yaml
+exec:
+  profile:
+    role:
+      ...
+```
+
+
+### Tokens
+No specific tokens are available.
+
+### Input
+Supported input types and default templates:
+
+#### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+#### Plain (`plain`)
+Default template:
+```
+{localpart}
+{domain}
+{mxid}
+```
+### Output
+Supported output types and default templates:
+
+#### JSON (`json`)
+Same as the [REST Identity Store](rest.md);
+
+#### Plain (`plain`)
+**Not supported at this time.** Use the JSON type.

docs/stores/firebase.md

@@ -0,0 +1,59 @@
+# Google Firebase Identity store
+https://firebase.google.com/
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | No         |
+| Identity       | Yes        |
+| Profile        | No         |
+
+## Requirements
+This backend requires a suitable Matrix client capable of performing Firebase authentication and passing the following
+information:
+- Firebase User ID as Matrix username
+- Firebase token as Matrix password
+
+If your client is Riot, you will need a custom version.
+
+## Configuration
+```yaml
+firebase:
+  enabled: <boolean>
+```
+Enable/disable this identity store.
+
+Example:
+```yaml
+firebase:
+  enabled: <boolean>
+```
+
+---
+
+```yaml
+firebase:
+  credentials: <string>
+```
+Path to the credentials file provided by Google Firebase to use with an external app.
+
+Example:
+```yaml
+firebase:
+  credentials: '/path/to/firebase/credentials.json'
+```
+
+---
+
+```yaml
+firebase:
+  database: <string>
+```
+URL to your Firebase database.
+
+Example:
+```yaml
+firebase:
+  database: 'https://my-project.firebaseio.com/'
+```

docs/stores/ldap.md

@@ -0,0 +1,139 @@
+# LDAP Identity store
+## Supported products:
+- Samba
+- Active Directory
+- OpenLDAP
+- NetIQ eDirectory
+
+For NetIQ, replace all the `ldap` prefix in the configuration by `netiq`.
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | Yes        |
+| Identity       | Yes        |
+| Profile        | Yes        |
+
+## Getting started
+### Base
+To use your LDAP backend, add the bare minimum configuration in mxisd config file:
+```yaml
+ldap:
+  enabled: true
+  connection:
+    host: 'ldapHostnameOrIp'
+    port: 389
+    bindDn: 'CN=My Mxisd User,OU=Users,DC=example,DC=org'
+    bindPassword: 'TheUserPassword'
+    baseDNs:
+      - 'OU=Users,DC=example,DC=org'
+```
+These are standard LDAP connection configuration. mxisd will try to connect on port default port 389 without encryption.
+
+If you would like to use several Base DNs, simply add more entries under `baseDNs`.
+
+### TLS/SSL connection
+If you would like to use a TLS/SSL connection, use the following configuration options (STARTLS not supported):
+```yaml
+ldap:
+  connection:
+    tls: true
+    port: 12345
+```
+
+### Filter results
+You can also set a default global filter on any LDAP queries:
+```yaml
+ldap:
+  filter: '(memberOf=CN=My Matrix Users,OU=Groups,DC=example,DC=org)'
+```
+This example would only return users part of the group called `My Matrix Users`.
+This can be overwritten or append in each specific flow describe below.
+
+For supported syntax, see the [LDAP library documentation](http://directory.apache.org/api/user-guide/2.3-searching.html#filter).
+
+### Attribute mapping
+LDAP features are based on mapping LDAP attributes to Matrix concepts, like a Matrix ID, its localpart, the user display
+name, their email(s) and/or phone number(s).
+     
+Default attributes are well suited for Active Directory/Samba. In case you are using a native LDAP backend, you will
+most certainly configure those mappings.
+
+#### User ID
+`ldap.attribute.uid.type`: How to process the User ID (UID) attribute:
+- `uid` will consider the value as the [Localpart](https://matrix.org/docs/spec/intro.html#user-identifiers)
+- `mxid` will consider the value as a complete [Matrix ID](https://matrix.org/docs/spec/intro.html#user-identifiers)
+
+`ldap.attribute.uid.value`: Attribute to use to set the User ID value.
+
+The following example would set the `sAMAccountName` attribute as a Matrix User ID localpart:
+```yaml
+ldap:
+  attribute:
+    uid:
+      type: 'uid'
+      value: 'sAMAccountName'
+```
+
+#### Display name
+Use `ldap.attribute.name`.
+
+The following example would set the display name to the value of the `cn` attribute:
+```yaml
+ldap:
+  attribute:
+    name: 'cn'
+```
+
+#### 3PIDs
+You can also change the attribute lists for 3PID, like email or phone numbers.
+
+The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67)
+for emails and phone number:
+```yaml
+ldap:
+  attribute:
+    threepid:
+      email:
+        - 'mail'
+        - 'otherMailAttribute'
+      msisdn:
+        - 'phone'
+        - 'otherPhoneAttribute'
+```
+
+## Features
+### Identity
+Identity features (related to 3PID invites or searches) are enabled and configured using default values and no specific
+configuration item is needed to get started.
+
+#### Configuration
+- `ldap.identity.filter`: Specific user filter applied during identity search. Global filter is used if blank/not set.
+- `ldap.identity.medium`: Namespace to overwrite generated queries from the list of attributes for each 3PID medium.
+
+### Authentication
+No further configuration is needed to use the Authentication feature with LDAP once globally enabled and configured.
+
+Profile auto-fill is enabled by default. It will use the `ldap.attribute.name` and `ldap.attribute.threepid` configuration
+options to get a lit of attributes to be used to build the user profile to pass on to synapse during authentication.
+
+#### Configuration
+- `ldap.auth.filter`: Specific user filter applied during identity search. Global filter is used if blank/not set.
+
+### Directory
+No further configuration is needed to use the Directory feature with LDAP once globally enabled and configured.
+
+#### Configuration
+To set a specific filter applied during directory search, use `ldap.directory.filter`
+
+If you would like to use extra attributes in search that are not 3PIDs, like nicknames, group names, employee number:
+```yaml
+ldap:
+  directory:
+    attribute:
+      other:
+        - 'myNicknameAttribute'
+        - 'memberOf'
+        - 'employeeNumberAttribute'
+```

docs/stores/rest.md

@@ -0,0 +1,277 @@
+# REST Identity store
+The REST backend allows you to query identity data in existing webapps, like:
+- Forums (phpBB, Discourse, etc.)
+- Custom Identity stores (Keycloak, ...)
+- CRMs (Wordpress, ...)
+- Self-hosted clouds (Nextcloud, ownCloud, ...)
+
+To integrate this backend with your webapp, you will need to implement the REST endpoints described below.
+
+## Features
+| Name                                            | Supported? |
+|-------------------------------------------------|------------|
+| [Authentication](../features/authentication.md) | Yes        |
+| [Directory](../features/directory.md)           | Yes        |
+| [Identity](../features/identity.md)             | Yes        |
+| [Profile](../features/profile.md)               | Yes        |
+
+## Configuration
+| Key                                  | Default                                        | Description                                          |
+|--------------------------------------|------------------------------------------------|------------------------------------------------------|
+| `rest.enabled`                       | `false`                                        | Globally enable/disable the REST backend             |
+| `rest.host`                          | *None*                                         | Default base URL to use for the different endpoints. |
+| `rest.endpoints.auth`                | `/_mxisd/backend/api/v1/auth/login`            | Validate credentials and get user profile            |
+| `rest.endpoints.directory`           | `/_mxisd/backend/api/v1/directory/user/search` | Search for users by arbitrary input                  |
+| `rest.endpoints.identity.single`     | `/_mxisd/backend/api/v1/identity/single`       | Endpoint to query a single 3PID                      |
+| `rest.endpoints.identity.bulk`       | `/_mxisd/backend/api/v1/identity/bulk`         | Endpoint to query a list of 3PID                     |
+| `rest.endpoints.profile.displayName` | `/_mxisd/backend/api/v1/profile/displayName`   | Query the display name for a Matrix ID
+| `rest.endpoints.profile.threepids`   | `/_mxisd/backend/api/v1/profile/threepids`     | Query the 3PIDs for a Matrix ID
+| `rest.endpoints.profile.roles`       | `/_mxisd/backend/api/v1/profile/roles`         | Query the Roles for a Matrix ID
+
+Endpoint values can handle two formats:
+- URL Path starting with `/` that gets happened to the `rest.host`
+- Full URL, if you want each endpoint to go to a specific server/protocol/port
+
+If an endpoint value is configured as an empty string, it will disable that specific feature, essentially bypassing the
+Identity store for that specific query.
+
+`rest.host` is mandatory if at least one endpoint is not a full URL.
+
+## Endpoints
+### Authentication
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
+  
+#### Request Body
+```json
+{
+  "auth": {
+    "mxid": "@john.doe:example.org",
+    "localpart": "john.doe",
+    "domain": "example.org",
+    "password": "passwordOfTheUser"
+  }
+}
+```
+
+#### Response Body
+If the authentication fails:
+```json
+{
+  "auth": {
+    "success": false
+  }
+}
+```
+
+If the authentication succeed:
+- `auth.id` supported values: `localpart`, `mxid`
+- `auth.profile` and any sub-member are all optional
+```json
+{
+  "auth": {
+    "success": true,
+    "id": {
+      "type": "localpart",
+      "value": "john"
+    },
+    "profile": {
+      "display_name": "John Doe",
+      "three_pids": [
+        {
+          "medium": "email",
+          "address": "john.doe@example.org"
+        },
+        {
+          "medium": "msisdn",
+          "address": "123456789"
+        }
+      ]
+    }
+  }
+}
+```
+
+### Directory
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
+
+#### Request Body
+```json
+{
+  "by": "<search type>",
+  "search_term": "doe"
+}
+```
+`by` can be:
+- `name`
+- `threepid`
+
+#### Response Body:
+If users found:
+```json
+{
+  "limited": false,
+  "results": [
+    {
+      "avatar_url": "http://domain.tld/path/to/avatar.png",
+      "display_name": "John Doe",
+      "user_id": "UserIdLocalpart"
+    },
+    {
+      "...": "..."
+    }
+  ]
+}
+```
+
+If no user found:
+```json
+{
+  "limited": false,
+  "results": []
+}
+```
+
+### Identity
+#### Single 3PID lookup
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
+  
+##### Request Body
+```json
+{
+  "lookup": {
+    "medium": "email",
+    "address": "john.doe@example.org"
+  }
+}
+```
+
+##### Response Body
+If a match was found:
+- `lookup.id.type` supported values: `localpart`, `mxid`
+```json
+{
+  "lookup": {
+    "medium": "email",
+    "address": "john.doe@example.org",
+    "id": {
+      "type": "mxid",
+      "value": "@john:example.org"
+    }
+  }
+}
+```
+
+If no match was found:
+```json
+{}
+```
+
+#### Bulk 3PID lookup
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
+  
+##### Request Body
+```json
+{
+  "lookup": [
+    {
+      "medium": "email",
+      "address": "john.doe@example.org"
+    },
+    {
+      "medium": "msisdn",
+      "address": "123456789"
+    }
+  ]
+}
+```
+
+##### Response Body
+For all entries where a match was found:
+- `lookup[].id.type` supported values: `localpart`, `mxid`
+```json
+{
+  "lookup": [
+    {
+      "medium": "email",
+      "address": "john.doe@example.org",
+      "id": {
+        "type": "localpart",
+        "value": "john"
+      }
+    },
+    {
+      "medium": "msisdn",
+      "address": "123456789",
+      "id": {
+        "type": "mxid",
+        "value": "@jane:example.org"
+      }
+    }
+  ]
+}
+```
+
+If no match was found:
+```json
+{
+  "lookup": []
+}
+```
+
+### Profile
+#### Request Body
+For all requests, the values are the same:
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
+
+With body (example values):
+##### Request Body
+```json
+{
+    "mxid": "@john.doe:example.org",
+    "localpart": "john.doe",
+    "domain": "example.org"
+}
+```
+#### Response Body
+For all responses, the same object structure will be parsed, making the non-relevant fields as optional.
+
+Structure with example values:
+```json
+{
+  "profile": {
+    "display_name": "John Doe",
+    "threepids": [
+      {
+        "medium": "email",
+        "address": "john.doe@example.org"
+      },
+      {
+        "...": "..."
+      }
+    ],
+    "roles": [
+      "DomainUsers",
+      "SalesOrg",
+      "..."
+    ]
+  }
+}
+```
+The base `profile` key is mandatory. `display_name`, `threepids` and `roles` are only to be returned on the relevant request.
+
+If there is no profile, the following response is expected:
+```json
+{
+  "profile": {}
+}
+```

docs/stores/sql.md

@@ -0,0 +1,110 @@
+# SQL Identity store
+## Supported Databases
+- PostgreSQL
+- MariaDB
+- MySQL
+- SQLite
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | No         |
+| Directory      | Yes        |
+| Identity       | Yes        |
+| Profile        | Yes        |
+
+Due to the implementation complexity of supporting arbitrary hashing/encoding mechanisms or auth flow, Authentication
+will be out of scope of SQL Identity stores and should be done via one of the other identity stores, typically
+the [Exec Identity Store](exec.md) or the [REST Identity Store](rest.md).
+
+## Configuration
+### Basic
+```yaml
+sql:
+  enabled: <boolean>
+```
+Enable/disable the identity store
+
+---
+
+```yaml
+sql:
+  type: <string>
+```
+Set the SQL backend to use:
+- `sqlite`
+- `postgresql`
+- `mariadb`
+- `mysql`
+
+### Connection
+#### SQLite
+```yaml
+sql:
+  connection: <string>
+```
+Set the value to the absolute path to the Synapse SQLite DB file.
+Example: `/path/to/sqlite/file.db`
+
+#### Others
+```yaml
+sql:
+  connection: //<HOST[:PORT]/DB?user=USER&password=PASS
+```
+Set the connection info for the database by replacing the following values:
+- `HOST`: Hostname of the SQL server
+- `PORT`: Optional port value, if not default
+- `DB`: Database name
+- `USER`: Username for the connection
+- `PASS`: Password for the connection
+
+This follow the JDBC URI syntax. See [official website](https://docs.oracle.com/javase/tutorial/jdbc/basics/connecting.html#db_connection_url).
+
+### Directory
+```yaml
+sql:
+  directory:
+    enabled: false
+```
+
+---
+
+```yaml
+sql:
+  directory:
+    query:
+      name:
+        type: <string>
+        value: <string>
+      threepid:
+        type: <string>
+        value: <string>
+```
+For each query, `type` can be used to tell mxisd how to process the ID column:
+- `localpart` will append the `matrix.domain` to it
+- `mxid` will use the ID as-is. If it is not a valid Matrix ID, the search will fail.
+
+`value` is the SQL query and must return two columns:
+- The first being the User ID
+- The second being its display name
+
+Example:
+```yaml
+sql:
+  directory:
+    query:
+      name:
+        type: 'localpart'
+        value: 'SELECT idColumn, displayNameColumn FROM table WHERE displayNameColumn LIKE ?'
+      threepid:
+        type: 'localpart'
+        value: 'SELECT idColumn, displayNameColumn FROM table WHERE threepidColumn LIKE ?'
+```
+
+### Identity
+```yaml
+sql:
+  identity:
+    type: <string>
+    query: <string>
+```

docs/stores/synapse.md

@@ -0,0 +1,53 @@
+# Synapse Identity Store
+Synapse's Database itself can be used as an Identity store.
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | No         |
+| Directory      | Yes        |
+| Identity       | Yes        |
+| Profile        | Yes        |
+
+Authentication is done by Synapse itself.
+
+## Configuration
+### Basic
+```yaml
+synapseSql:
+  enabled: <boolean>
+```
+Enable/disable the identity store
+
+---
+
+```yaml
+synapseSql:
+  type: <string>
+```
+Set the SQL backend to use which is configured in synapse:
+- `sqlite`
+- `postgresql`
+
+### SQLite
+```yaml
+synapseSql:
+  connection: <string>
+```
+Set the value to the absolute path to the Synapse SQLite DB file.
+Example: `/path/to/synapse/sqliteFile.db`
+
+### PostgreSQL
+```yaml
+synapseSql:
+  connection: //<HOST[:PORT]/DB?user=USER&password=PASS
+```
+Set the connection info for the database by replacing the following values:
+- `HOST`: Hostname of the SQL server
+- `PORT`: Optional port value, if not default
+- `DB`: Database name
+- `USER`: Username for the connection
+- `PASS`: Password for the connection
+
+### Query customization
+See the [SQL Identity store](sql.md)

docs/stores/wordpress.md

@@ -0,0 +1,75 @@
+# Wordpress Identity store
+This Identity store allows you to use user accounts registered on your Wordpress setup.  
+Two types of connections are required for full support:
+- [REST API](https://developer.wordpress.org/rest-api/) with JWT authentication
+- Direct SQL access
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | Yes        |
+| Identity       | Yes        |
+| Profile        | No         |
+
+## Requirements
+- [Wordpress](https://wordpress.org/download/) >= 4.4
+- Permalink structure set to `Post Name`
+- [JWT Auth plugin for REST API](https://wordpress.org/plugins/jwt-authentication-for-wp-rest-api/)
+- SQL Credentials to the Wordpress Database
+
+## Configuration
+### Wordpress
+#### JWT Auth
+Set a JWT secret into `wp-config.php` like so:
+```php
+define('JWT_AUTH_SECRET_KEY', 'your-top-secret-key');
+```
+`your-top-secret-key` should be set to a randomly generated value which is kept secret.
+
+#### Rewrite of `index.php`
+Wordpress is normally configured with rewrite of `index.php` so it does not appear in URLs.  
+If this is not the case for your installation, the mxisd URL will need to be appended with `/index.php`
+
+### mxisd
+Enable in the configuration:
+```yaml
+wordpress:
+  enabled: true
+```
+Configure the URL to your Wordpress installation - see above about added `/index.php`:
+```yaml
+wordpress:
+  rest:
+    base: 'http://localhost:8080'
+```
+Configure the SQL connection to your Wordpress database:
+```yaml
+wordpress:
+  sql:
+    connection: '//127.0.0.1/wordpress?user=root&password=example'
+```
+
+---
+
+By default, MySQL database is expected. If you use another database, use:
+```yaml
+wordpress:
+  sql:
+    type: <string>
+```
+With possible values:
+- `mysql`
+- `mariadb`
+- `postgresql`
+- `sqlite`
+
+---
+
+To configure the tables prefix for default queries, in case a custom value was set during Wordpress install:
+```yaml
+wordpress:
+  sql:
+    tablePrefix: <string>
+```
+By default, the value is set to `wp_`.

docs/threepids/medium/email/smtp-connector.md

@@ -0,0 +1,21 @@
+# Email notifications - SMTP connector
+Enabled by default.
+
+Connector ID: `smtp`
+
+## Configuration
+```yaml
+threepid:
+  medium:
+    email:
+      identity:
+        from: 'identityServerEmail@example.org'
+        name: 'My Identity Server'
+      connectors:
+        smtp:
+          host: 'smtpHostname'
+          port: 587
+          tls: 1 # 0 = no STARTLS, 1 = try, 2 = force
+          login: 'smtpLogin'
+          password: 'smtpPassword'
+```

docs/threepids/medium/msisdn/twilio-connector.md

@@ -0,0 +1,16 @@
+# SMS notifications - Twilio connector
+Enabled by default.
+
+Connector ID: `twilio`
+
+## Configuration
+```yaml
+threepid:
+  medium:
+    msisdn:
+      connectors:
+        twilio:
+          accountSid: 'myAccountSid'
+          authToken: 'myAuthToken'
+          number: '+123456789'
+```

docs/threepids/notification/basic-handler.md

@@ -0,0 +1,40 @@
+# Basic Notification handler
+Basic notification handler which uses two components:
+- Content generator, to produce the notifications
+- Connectors to send the notification content
+
+This handler can be used with the 3PID types:
+- `email`
+- `msisdn` (Phone numbers)
+
+## Generators
+- [Template](template-generator.md)
+## Connectors
+- Email
+  - [SMTP](../medium/email/smtp-connector.md)
+- SMS
+  - [Twilio](../medium/msisdn/twilio-connector.md)
+
+## Configuration
+Enabled by default or with:
+```yaml
+notification:
+  handler:
+    email: 'raw'
+```
+
+**WARNING:** Will be consolidated soon, prone to breaking changes.  
+Structure and default values:
+```yaml
+threepid:
+  medium:
+    email:
+      identity:
+        from: ''
+        name: ''
+      connector: 'smtp'
+      generator: 'template'
+    msisdn:
+      connector: 'twilio'
+      generator: 'template'
+```

docs/threepids/notification/sendgrid-handler.md

@@ -0,0 +1,45 @@
+# SendGrid Notification handler
+> **WARNING:** This section is incomplete and may be misleading. Contact us if guidance is needed.
+
+Enable with:
+```yaml
+notification:
+  handler:
+    email: 'sendgrid'
+```
+
+Available Configuration keys:
+```yaml
+notification:
+  handlers:
+    sendgrid:
+      api:
+        key: <API key>
+      identity:
+        from: <Sender email address>
+        name: <Sender name>
+      templates:
+        invite:
+          subject: <Subject of the email notification sent for room invites>
+          body:
+            text: <Path to file containing the raw text part of the email. Do not set to not use one>
+            html: <Path to file containing the HTML part of the email. Do not set to not use one>
+        session:
+          validation:
+            local:
+              subject: <Subject of the email notification sent for local 3PID sessions>
+              body:
+                text: <Path to file containing the raw text part of the email. Do not set to not use one>
+                html: <Path to file containing the HTML part of the email. Do not set to not use one>
+            remote:
+              subject: <Subject of the email notification sent for remote 3PID sessions>
+              body:
+                text: <Path to file containing the raw text part of the email. Do not set to not use one>
+                html: <Path to file containing the HTML part of the email. Do not set to not use one>
+          unbind:
+            fraudulent:
+              subject: <Subject of the email notification sent for potentially fraudulent 3PID unbinds>
+              body:
+                text: <Path to file containing the raw text part of the email. Do not set to not use one>
+                html: <Path to file containing the raw text part of the email. Do not set to not use one>
+``` 

docs/threepids/notification/template-generator.md

@@ -0,0 +1,77 @@
+# Notifications: Generate from templates
+To create notification content, you can use the `template` generator if supported for the 3PID medium which will read
+content from configured files.
+
+Placeholders can be integrated into the templates to dynamically populate such content with relevant information like
+the 3PID that was requested, the domain of your Identity server, etc.
+
+Templates can be configured for each event that would send a notification to the end user. Events share a set of common
+placeholders and also have their own individual set of placeholders.
+
+## Configuration
+To configure paths to the various templates:
+```yaml
+threepid:
+  medium:
+    <YOUR 3PID MEDIUM HERE>:
+      generators:
+        template:
+          invite: '/path/to/invite-template.eml'
+          session:
+            validation:
+              local: '/path/to/validate-local-template.eml'
+              remote: '/path/to/validate-remote-template.eml'
+            unbind:
+              frandulent: '/path/to/unbind-fraudulent-template.eml'
+          generic:
+            matrixId: '/path/to/mxid-invite-template.eml'
+```
+The `template` generator is usually the default, so no further configuration is needed.
+
+##  Global placeholders
+| Placeholder           | Purpose                                                                      |
+|-----------------------|------------------------------------------------------------------------------|
+| `%DOMAIN%`            | Identity server authoritative domain, as configured in `matrix.domain`       |
+| `%DOMAIN_PRETTY%`     | Same as `%DOMAIN%` with the first letter upper case and all other lower case |
+| `%FROM_EMAIL%`        | Email address configured in `threepid.medium.<3PID medium>.identity.from`    |
+| `%FROM_NAME%`         | Name configured in `threepid.medium.<3PID medium>.identity.name`             |
+| `%RECIPIENT_MEDIUM%`  | The 3PID medium, like `email` or `msisdn`                                    |
+| `%RECIPIENT_ADDRESS%` | The address to which the notification is sent                                |
+
+## Events
+### Room invitation
+This template is used when someone is invited into a room using an email address which has no known bind to a Matrix ID.
+#### Placeholders
+| Placeholder           | Purpose                                                                                  |
+|-----------------------|------------------------------------------------------------------------------------------|
+| `%SENDER_ID%`         | Matrix ID of the user who made the invite                                                |
+| `%SENDER_NAME%`       | Display name of the user who made the invite, if not available/set, empty                |
+| `%SENDER_NAME_OR_ID%` | Display name of the user who made the invite. If not available/set, its Matrix ID        |
+| `%INVITE_MEDIUM%`     | The 3PID medium for the invite.                                                          |
+| `%INVITE_ADDRESS%`    | The 3PID address for the invite.                                                         |
+| `%ROOM_ID%`           | The Matrix ID of the Room in which the invite took place                                 |
+| `%ROOM_NAME%`         | The Name of the room in which the invite took place. If not available/set, empty         |
+| `%ROOM_NAME_OR_ID%`   | The Name of the room in which the invite took place. If not available/set, its Matrix ID |
+
+### Local validation of 3PID Session
+This template is used when to user which added their 3PID address to their profile/settings and the session policy
+allows at least local sessions.  
+
+#### Placeholders
+| Placeholder          | Purpose                                                                              |
+|----------------------|--------------------------------------------------------------------------------------|
+| `%VALIDATION_LINK%`  | URL, including token, to validate the 3PID session.                                  |
+| `%VALIDATION_TOKEN%` | The token needed to validate the local session, in case the user cannot use the link |
+
+### Remote validation of 3PID Session
+This template is used when to user which added their 3PID address to their profile/settings and the session policy only
+allows remote sessions.
+
+**NOTE:** 3PID session always require local validation of a token, even if a remote session is enforced.  
+One cannot bind a Matrix ID to the session until both local and remote sessions have been validated.
+
+#### Placeholders
+| Placeholder          | Purpose                                                |
+|----------------------|--------------------------------------------------------|
+| `%VALIDATION_TOKEN%` | The token needed to validate the session               |
+| `%NEXT_URL%`         | URL to continue with remote validation of the session. |

docs/threepids/session/session-views.md

@@ -0,0 +1,34 @@
+# Web pages for the 3PID sessions
+You can customize the various pages used during a 3PID validation using the options below.
+
+## Configuration
+Pseudo-configuration to illustrate the structure:
+```yaml
+# CONFIGURATION EXAMPLE
+# DO NOT COPY/PASTE THIS IN YOUR CONFIGURATION
+view:
+  session:
+    onTokenSubmit:
+      success: '/path/to/session/tokenSubmitSuccess-page.html'
+      failure: '/path/to/session/tokenSubmitFailure-page.html'
+# CONFIGURATION EXAMPLE
+# DO NOT COPY/PASTE THIS IN YOUR CONFIGURATION
+```
+
+`view.session`:
+This is triggered when a user submit a validation token for a 3PID session. It is typically visited when clicking the
+link in a validation email.
+
+The template should typically inform the user that the validation was successful and to go back in their Matrix client
+to finish the validation process, or that the validation failed.
+
+Two configuration keys are available that accept paths to HTML templates:
+- `success`
+- `failure`
+
+## Placeholders
+### Success
+No object/placeholder are currently available.
+
+### Failure
+No object/placeholder are currently available.

docs/threepids/session/session.md

@@ -0,0 +1,147 @@
+# 3PID Sessions
+- [Overview](#overview)
+- [Restrictions](#restrictions)
+  - [Bindings](#bindings)
+  - [Federation](#federation)
+- [Notifications](#notifications)
+  - [Email](#email)
+  - [Phone numbers](#msisdn-(phone-numbers))
+- [Usage](#usage)
+  - [Configuration](#configuration)
+  - [Web views](#web-views)
+  - [Scenarios](#scenarios)
+    - [Sessions disabled](#sessions-disabled)
+
+## Overview
+When adding an email, a phone number or any other kind of 3PID (Third-Party Identifier) in a Matrix client,
+the identity server is contacted to validate the 3PID.
+
+To validate the 3PID, the identity server creates a session associated with a secret token. That token is sent via a message
+to the 3PID (e.g. an email) with a the necessary info so the user can submit them to the Identity Server, confirm ownership
+of the 3PID.
+
+Once this 3PID is validated, the Homeserver will request that the Identity Server links the provided user Matrix ID with
+the 3PID session and finally add the 3PID to its own data store.
+
+This serves two purposes:
+- Add the 3PID as an administrative/login info for the Homeserver directly
+- Links, called *Bind*, the 3PID so it can be queried from Homeservers and clients when inviting someone in a room
+by a 3PID, allowing it to be resolved to a Matrix ID.
+
+## Restrictions
+### Bindings
+mxisd does not store bindings directly. While a user can see its email, phone number or any other 3PID in its
+settings/profile, it does **NOT** mean it is published/saved anywhere or can be used to invite/search the user.
+
+Identity stores are the ones holding such data, irrelevant if a user added a 3PID to their profile. When queried for
+bindings, mxisd will query Identity stores which are responsible to store this kind of information.
+
+Therefore, by default, any 3PID added to a user profile which is NOT within a configured and enabled Identity backend
+will simply not be usable for search or invites, **even on the same Homeserver!**  
+
+To have such 3PID bindings available for search and invite queries on synapse, use its dedicated
+[Identity store](../../stores/synapse.md).
+
+### Federation
+In a federated set up, identity servers must cooperate to find the Matrix ID associated with a 3PID.
+
+Federation is based on the principle that each server is responsible for its own (dns) domain.
+Therefore only those 3PID can be federated that can be distinguished by their
+domain such as email addresses.
+
+Example: a user from Homeserver `example.org` adds an email `john@example.com`.  
+Federated identity servers would try to find the identity server at `example.com` and ask it for the Matrix ID of associated with `john@example.com`.
+
+Nevertheless, Matrix users might add 3PIDs that are not associated to a domain, for example telephone numbers.
+Or they might even add 3PIDs associated to a different domain (such as an email address hosted by Gmail).
+Such 3PIDs cannot be resolved in a federated way and will not be found from other servers.
+
+Example: a user from Homeserver `example.org` adds an email `john@gmail.com`.  
+If a federated lookup was performed, Identity servers would try to find the 3PID bind at the `gmail.com` server, and
+not `example.org`.
+
+As mxisd is built for self-hosted use cases, mainly for orgs/corps, this is usually not a problem for emails.  
+Sadly, there is currently no mechanism to make this work for phone numbers. 
+
+## Notifications
+3PIDs are validated by sending a pre-formatted message containing a token to that 3PID address, which must be given to the
+Identity server that received the request. This is usually done by means of a URL to visit for email or a short number
+received by SMS for phone numbers.
+
+mxisd use two components for this:
+- Generator which produces the message to be sent with the necessary information the user needs to validate their session.
+- Connector which actually send the notification (e.g. SMTP for email).
+
+Built-in generators and connectors for supported 3PID types:
+
+### Email
+Generators:
+- [Template](../notification/template-generator.md)
+
+Connectors:
+- [SMTP](../medium/email/smtp-connector.md)
+
+#### MSISDN (Phone numbers)
+Generators:
+- [Template](../notification/template-generator.md)
+
+Connectors:
+ - [Twilio](../medium/msisdn/twilio-connector.md) with SMS
+
+## Usage
+### Configuration
+The following example of configuration shows which items are relevant for 3PID sessions.
+
+**IMPORTANT:** Most configuration items shown have default values and should not be included in your own configuration
+file unless you want to specifically overwrite them.
+```yaml
+# CONFIGURATION EXAMPLE
+# DO NOT COPY/PASTE AS-IS IN YOUR CONFIGURATION
+
+session:
+  policy:
+    validation:
+      enabled: true
+    unbind:
+      fraudulent:
+        sendWarning: true
+
+# DO NOT COPY/PASTE AS-IS IN YOUR CONFIGURATION
+# CONFIGURATION EXAMPLE
+```
+
+`session.policy.validation` is the core configuration to control what users configured to use your Identity server
+are allowed to do in terms of 3PID sessions. The policy has a global on/off switch for 3PID sessions using `.enabled`  
+
+---
+
+`unbind.fraudulent` controls warning notifications if an illegal/fraudulent 3PID removal is attempted on the Identity server.  
+This is directly related to synapse disregard for privacy and new GDPR laws in Europe in an attempt to inform users about
+potential privacy leaks.
+
+For more information, see the corresponding [synapse issue](https://github.com/matrix-org/synapse/issues/4540).
+
+### Web views
+Once a user click on a validation link, it is taken to the Identity Server validation page where the token is submitted.  
+If the session or token is invalid, an error page is displayed.  
+Workflow pages are also available for the remote 3PID session process.
+
+See [the dedicated document](session-views.md)
+on how to configure/customize/brand those pages to your liking.
+
+### Scenarios
+#### Sessions disabled
+This configuration would disable 3PID sessions altogether, preventing users from validating emails and/or phone numbers
+and any subsequent actions that requires them, like adding them to their profiles.
+  
+This would be used if mxisd is also performing authentication for the Homeserver, typically with synapse and the
+[REST password provider](https://github.com/kamax-matrix/matrix-synapse-rest-auth), where 3PID mappings would be
+auto-populated.
+
+Use the following values to enable this mode:
+```yaml
+session:
+  policy:
+    validation:
+      enabled: false
+```

mxisd.example.yaml

@@ -0,0 +1,116 @@
+# Sample configuration file explaining the minimum required keys to be set to run mxisd
+#
+# For a complete list of options, see https://github.com/kamax-matrix/mxisd/docs/README.md
+
+#######################
+# Matrix config items #
+#######################
+# Matrix domain, same as the domain configure in your Homeserver configuration.
+# NOTE: in Synapse Homeserver, the Matrix domain is defined as 'server_name' in configuration file.
+#
+# This is used to build the various identifiers in all the features.
+matrix:
+  domain: ''
+
+
+################
+# Signing keys #
+################
+# Absolute path for the Identity Server signing key.
+# This is **NOT** your homeserver key.
+# The signing key is auto-generated during execution time if not present.
+#
+# During testing, /var/tmp/mxisd.key is a possible value
+# For production, recommended location shall be one of the following:
+#   - /var/opt/mxisd/sign.key
+#   - /var/local/mxisd/sign.key
+#   - /var/lib/mxisd/sign.key
+#
+key:
+  path: ''
+
+
+# Path to the SQLite DB file for mxisd internal storage
+#
+# Examples:
+#  - /var/opt/mxisd/mxisd.db
+#  - /var/local/mxisd/mxisd.db
+#  - /var/lib/mxisd/mxisd.db
+#
+storage:
+  provider:
+    sqlite:
+      database: '/path/to/mxisd.db'
+
+
+####################
+# Fallback servers #
+####################
+#
+# Root/Central servers to be used as final fallback when performing lookups.
+# By default, for privacy reasons, matrix.org servers are not enabled.
+# See the following issue: https://github.com/kamax-matrix/mxisd/issues/76
+#
+# If you would like to use them and trade away your privacy for convenience, uncomment the following option:
+#
+#forward:
+#  servers: ['matrix-org']
+
+
+################
+# LDAP Backend #
+################
+# If you would like to integrate with your AD/Samba/LDAP server,
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/ldap.md
+
+
+###############
+# SQL Backend #
+###############
+# If you would like to integrate with a MySQL/MariaDB/PostgreQL/SQLite DB,
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/sql.md
+
+
+################
+# REST Backend #
+################
+# If you would like to integrate with an existing web service/webapp,
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/stores/rest.md
+
+
+#################################################
+# Notifications for invites/addition to profile #
+#################################################
+# If you would like to change the content,
+# see https://github.com/kamax-matrix/mxisd/blob/master/docs/threepids/notification/template-generator.md
+#
+#### E-mail invite sender
+threepid:
+  medium:
+    email:
+      identity:
+        # The e-mail to send as.
+        from: "matrix-identity@example.org"
+
+      connectors:
+        smtp:
+          # SMTP host
+          host: "smtp.example.org"
+
+          # SMTP port
+          port: 587
+
+          # TLS mode for the connection.
+          #
+          # Possible values:
+          #  0    Disable TLS entirely
+          #  1    Enable TLS if supported by server (default)
+          #  2    Force TLS and fail if not available
+          #
+          tls: 1
+
+          # Login for SMTP
+          login: "matrix-identity@example.org"
+
+          # Password for the account
+          password: "ThePassword"

src/debian/control

@@ -1,6 +1,6 @@
 Package: mxisd
 Maintainer: Kamax.io <foss@kamax.io>
-Homepage: https://github.com/kamax-io/mxisd
+Homepage: https://github.com/kamax-matrix/mxisd
 Description: Federated Matrix Identity Server
 Architecture: all
 Depends: openjdk-8-jre | openjdk-8-jre-headless | openjdk-8-jdk | openjdk-8-jdk-headless

src/debian/postinst

@@ -6,8 +6,8 @@ useradd -r mxisd || true
 # Set permissions for data directory
 chown -R mxisd:mxisd %DEB_DATA_DIR%
 
-# Create symlink to mxusd
+# Create symlink to mxisd run script
-ln -sfT /usr/lib/mxisd/mxisd.jar /usr/bin/mxisd
+ln -sfT /usr/lib/mxisd/mxisd /usr/bin/mxisd
 
 # Enable systemd service
 systemctl enable mxisd.service

src/docker/start.sh

@@ -1,2 +1,34 @@
-#!/bin/sh
+#!/bin/bash
-exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.config.location=/etc/mxisd/ -Dspring.config.name=mxisd -jar /mxisd.jar
+
+if [[ -n "$CONF_FILE_PATH" ]] && [ ! -f "$CONF_FILE_PATH" ]; then
+    echo "Generating config file $CONF_FILE_PATH"
+    touch "CONF_FILE_PATH"
+
+    if [[ -n "$MATRIX_DOMAIN" ]]; then
+        echo "Setting matrix domain to $MATRIX_DOMAIN"
+        echo "matrix:" >> "$CONF_FILE_PATH"
+        echo "  domain: '$MATRIX_DOMAIN'" >> "$CONF_FILE_PATH"
+        echo >> "$CONF_FILE_PATH"
+    fi
+
+    if [[ -n "$SIGN_KEY_PATH" ]]; then
+        echo "Setting signing key path to $SIGN_KEY_PATH"
+        echo "key:" >> "$CONF_FILE_PATH"
+        echo "  path: '$SIGN_KEY_PATH'" >> "$CONF_FILE_PATH"
+        echo >> "$CONF_FILE_PATH"
+    fi
+
+    if [[ -n "$SQLITE_DATABASE_PATH" ]]; then
+        echo "Setting SQLite DB path to $SQLITE_DATABASE_PATH"
+        echo "storage:" >> "$CONF_FILE_PATH"
+        echo "  provider:" >> "$CONF_FILE_PATH"
+        echo "    sqlite:" >> "$CONF_FILE_PATH"
+        echo "      database: '$SQLITE_DATABASE_PATH'" >> "$CONF_FILE_PATH"
+        echo >> "$CONF_FILE_PATH"
+    fi
+
+    echo "Starting mxisd..."
+    echo
+fi
+
+exec java -jar /app/mxisd.jar -c /etc/mxisd/mxisd.yaml

src/main/groovy/io/kamax/mxisd/auth/AuthManager.java

@@ -1,93 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.auth;
-
-import io.kamax.matrix.MatrixID;
-import io.kamax.matrix._MatrixID;
-import io.kamax.mxisd.ThreePid;
-import io.kamax.mxisd.UserIdType;
-import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
-import io.kamax.mxisd.auth.provider.BackendAuthResult;
-import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.invitation.InvitationManager;
-import io.kamax.mxisd.lookup.ThreePidMapping;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Service;
-
-import java.util.ArrayList;
-import java.util.List;
-
-@Service
-public class AuthManager {
-
-    private Logger log = LoggerFactory.getLogger(AuthManager.class);
-
-    @Autowired
-    private List<AuthenticatorProvider> providers = new ArrayList<>();
-
-    @Autowired
-    private MatrixConfig mxCfg;
-
-    @Autowired
-    private InvitationManager invMgr;
-
-    public UserAuthResult authenticate(String id, String password) {
-        _MatrixID mxid = new MatrixID(id);
-        for (AuthenticatorProvider provider : providers) {
-            if (!provider.isEnabled()) {
-                continue;
-            }
-
-            BackendAuthResult result = provider.authenticate(mxid, password);
-            if (result.isSuccess()) {
-
-                String mxId;
-                if (UserIdType.Localpart.is(result.getId().getType())) {
-                    mxId = new MatrixID(result.getId().getValue(), mxCfg.getDomain()).getId();
-                } else if (UserIdType.MatrixID.is(result.getId().getType())) {
-                    mxId = new MatrixID(result.getId().getValue()).getId();
-                } else {
-                    log.warn("Unsupported User ID type {} for backend {}", result.getId().getType(), provider.getClass().getSimpleName());
-                    continue;
-                }
-
-                UserAuthResult authResult = new UserAuthResult().success(mxId, result.getProfile().getDisplayName());
-                for (ThreePid pid : result.getProfile().getThreePids()) {
-                    authResult.withThreePid(pid.getMedium(), pid.getAddress());
-                }
-                log.info("{} was authenticated by {}, publishing 3PID mappings, if any", id, provider.getClass().getSimpleName());
-                for (ThreePid pid : authResult.getThreePids()) {
-                    log.info("Processing {} for {}", pid, id);
-                    invMgr.publishMappingIfInvited(new ThreePidMapping(pid, authResult.getMxid()));
-                }
-
-                invMgr.lookupMappingsForInvites();
-
-                return authResult;
-            }
-        }
-
-        return new UserAuthResult().failure();
-    }
-
-}

src/main/groovy/io/kamax/mxisd/backend/firebase/GoogleFirebaseAuthenticator.groovy

@@ -1,193 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.backend.firebase
-
-import com.google.firebase.FirebaseApp
-import com.google.firebase.FirebaseOptions
-import com.google.firebase.auth.*
-import com.google.firebase.internal.NonNull
-import com.google.firebase.tasks.OnFailureListener
-import com.google.firebase.tasks.OnSuccessListener
-import io.kamax.matrix.ThreePidMedium
-import io.kamax.matrix._MatrixID
-import io.kamax.mxisd.ThreePid
-import io.kamax.mxisd.UserIdType
-import io.kamax.mxisd.auth.provider.AuthenticatorProvider
-import io.kamax.mxisd.auth.provider.BackendAuthResult
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-
-import java.util.concurrent.CountDownLatch
-import java.util.concurrent.TimeUnit
-import java.util.regex.Pattern
-
-public class GoogleFirebaseAuthenticator implements AuthenticatorProvider {
-
-    private Logger log = LoggerFactory.getLogger(GoogleFirebaseAuthenticator.class);
-
-    private static final Pattern matrixIdLaxPattern = Pattern.compile("@(.*):(.+)"); // FIXME use matrix-java-sdk
-
-    private boolean isEnabled;
-    private String domain;
-    private FirebaseApp fbApp;
-    private FirebaseAuth fbAuth;
-
-    private void waitOnLatch(BackendAuthResult result, CountDownLatch l, long timeout, TimeUnit unit, String purpose) {
-        try {
-            l.await(timeout, unit);
-        } catch (InterruptedException e) {
-            log.warn("Interrupted while waiting for " + purpose);
-            result.failure();
-        }
-    }
-
-    public GoogleFirebaseAuthenticator(boolean isEnabled) {
-        this.isEnabled = isEnabled;
-    }
-
-    public GoogleFirebaseAuthenticator(String credsPath, String db, String domain) {
-        this(true);
-        this.domain = domain;
-        try {
-            fbApp = FirebaseApp.initializeApp(getOpts(credsPath, db), "AuthenticationProvider");
-            fbAuth = FirebaseAuth.getInstance(fbApp);
-
-            log.info("Google Firebase Authentication is ready");
-        } catch (IOException e) {
-            throw new RuntimeException("Error when initializing Firebase", e);
-        }
-    }
-
-    private FirebaseCredential getCreds(String credsPath) throws IOException {
-        if (StringUtils.isNotBlank(credsPath)) {
-            return FirebaseCredentials.fromCertificate(new FileInputStream(credsPath));
-        } else {
-            return FirebaseCredentials.applicationDefault();
-        }
-    }
-
-    private FirebaseOptions getOpts(String credsPath, String db) throws IOException {
-        if (StringUtils.isBlank(db)) {
-            throw new IllegalArgumentException("Firebase database is not configured");
-        }
-
-        return new FirebaseOptions.Builder()
-                .setCredential(getCreds(credsPath))
-                .setDatabaseUrl(db)
-                .build();
-    }
-
-    @Override
-    public boolean isEnabled() {
-        return isEnabled;
-    }
-
-    private void waitOnLatch(CountDownLatch l) {
-        try {
-            l.await(30, TimeUnit.SECONDS);
-        } catch (InterruptedException e) {
-            log.warn("Interrupted while waiting for Firebase auth check");
-        }
-    }
-
-    @Override
-    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
-        if (!isEnabled()) {
-            throw new IllegalStateException();
-        }
-
-        log.info("Trying to authenticate {}", mxid);
-
-        BackendAuthResult result = BackendAuthResult.failure();
-
-        String localpart = m.group(1);
-        CountDownLatch l = new CountDownLatch(1);
-        fbAuth.verifyIdToken(password).addOnSuccessListener(new OnSuccessListener<FirebaseToken>() {
-            @Override
-            void onSuccess(FirebaseToken token) {
-                try {
-                    if (!StringUtils.equals(localpart, token.getUid())) {
-                        log.info("Failture to authenticate {}: Matrix ID localpart '{}' does not match Firebase UID '{}'", id, localpart, token.getUid());
-                        result = BackendAuthResult.failure();
-                        return;
-                    }
-
-                    result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, token.getName());
-                    log.info("{} was successfully authenticated", mxid);
-                    log.info("Fetching profile for {}", mxid);
-                    CountDownLatch userRecordLatch = new CountDownLatch(1);
-                    fbAuth.getUser(token.getUid()).addOnSuccessListener(new OnSuccessListener<UserRecord>() {
-                        @Override
-                        void onSuccess(UserRecord user) {
-                            try {
-                                if (StringUtils.isNotBlank(user.getEmail())) {
-                                    result.withThreePid(new ThreePid(ThreePidMedium.Email.getId(), user.getEmail()));
-                                }
-
-                                if (StringUtils.isNotBlank(user.getPhoneNumber())) {
-                                    result.withThreePid(new ThreePid(ThreePidMedium.PhoneNumber.getId(), user.getPhoneNumber()));
-                                }
-
-                            } finally {
-                                userRecordLatch.countDown();
-                            }
-                        }
-                    }).addOnFailureListener(new OnFailureListener() {
-                        @Override
-                        void onFailure(@NonNull Exception e) {
-                            try {
-                                log.warn("Unable to fetch Firebase user profile for {}", mxid);
-                                result = BackendAuthResult.failure();
-                            } finally {
-                                userRecordLatch.countDown();
-                            }
-                        }
-                    });
-
-                    waitOnLatch(result, userRecordLatch, 30, TimeUnit.SECONDS, "Firebase user profile");
-                } finally {
-                    l.countDown()
-                }
-            }
-        }).addOnFailureListener(new OnFailureListener() {
-            @Override
-            void onFailure(@NonNull Exception e) {
-                try {
-                    if (e instanceof IllegalArgumentException) {
-                        log.info("Failure to authenticate {}: invalid firebase token", mxid);
-                    } else {
-                        log.info("Failure to authenticate {}: {}", id, e.getMessage(), e);
-                        log.info("Exception", e);
-                    }
-
-                    result = BackendAuthResult.failure();
-                } finally {
-                    l.countDown()
-                }
-            }
-        });
-
-        waitOnLatch(result, l, 30, TimeUnit.SECONDS, "Firebase auth check");
-        return result;
-    }
-
-}

src/main/groovy/io/kamax/mxisd/backend/firebase/GoogleFirebaseProvider.groovy

@@ -1,191 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.backend.firebase
-
-import com.google.firebase.FirebaseApp
-import com.google.firebase.FirebaseOptions
-import com.google.firebase.auth.FirebaseAuth
-import com.google.firebase.auth.FirebaseCredential
-import com.google.firebase.auth.FirebaseCredentials
-import com.google.firebase.auth.UserRecord
-import com.google.firebase.internal.NonNull
-import com.google.firebase.tasks.OnFailureListener
-import com.google.firebase.tasks.OnSuccessListener
-import io.kamax.matrix.ThreePidMedium
-import io.kamax.mxisd.lookup.SingleLookupReply
-import io.kamax.mxisd.lookup.SingleLookupRequest
-import io.kamax.mxisd.lookup.ThreePidMapping
-import io.kamax.mxisd.lookup.provider.IThreePidProvider
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-
-import java.util.concurrent.CountDownLatch
-import java.util.concurrent.TimeUnit
-import java.util.function.Consumer
-import java.util.regex.Pattern
-
-public class GoogleFirebaseProvider implements IThreePidProvider {
-
-    private Logger log = LoggerFactory.getLogger(GoogleFirebaseProvider.class);
-
-    private static final Pattern matrixIdLaxPattern = Pattern.compile("@(.*):(.+)");
-
-    private boolean isEnabled;
-    private String domain;
-    private FirebaseApp fbApp;
-    private FirebaseAuth fbAuth;
-
-    public GoogleFirebaseProvider(boolean isEnabled) {
-        this.isEnabled = isEnabled;
-    }
-
-    public GoogleFirebaseProvider(String credsPath, String db, String domain) {
-        this(true);
-        this.domain = domain;
-        try {
-            fbApp = FirebaseApp.initializeApp(getOpts(credsPath, db), "ThreePidProvider");
-            fbAuth = FirebaseAuth.getInstance(fbApp);
-
-            log.info("Google Firebase Authentication is ready");
-        } catch (IOException e) {
-            throw new RuntimeException("Error when initializing Firebase", e);
-        }
-    }
-
-    private FirebaseCredential getCreds(String credsPath) throws IOException {
-        if (StringUtils.isNotBlank(credsPath)) {
-            return FirebaseCredentials.fromCertificate(new FileInputStream(credsPath));
-        } else {
-            return FirebaseCredentials.applicationDefault();
-        }
-    }
-
-    private FirebaseOptions getOpts(String credsPath, String db) throws IOException {
-        if (StringUtils.isBlank(db)) {
-            throw new IllegalArgumentException("Firebase database is not configured");
-        }
-
-        return new FirebaseOptions.Builder()
-                .setCredential(getCreds(credsPath))
-                .setDatabaseUrl(db)
-                .build();
-    }
-
-    private String getMxid(UserRecord record) {
-        return "@${record.getUid()}:${domain}";
-    }
-
-    @Override
-    public boolean isEnabled() {
-        return isEnabled;
-    }
-
-    @Override
-    public boolean isLocal() {
-        return true;
-    }
-
-    @Override
-    public int getPriority() {
-        return 25;
-    }
-
-    private void waitOnLatch(CountDownLatch l) {
-        try {
-            l.await(30, TimeUnit.SECONDS);
-        } catch (InterruptedException e) {
-            log.warn("Interrupted while waiting for Firebase auth check");
-        }
-    }
-
-    private Optional<UserRecord> findInternal(String medium, String address) {
-        UserRecord r;
-        CountDownLatch l = new CountDownLatch(1);
-
-        OnSuccessListener<UserRecord> success = new OnSuccessListener<UserRecord>() {
-            @Override
-            void onSuccess(UserRecord result) {
-                log.info("Found 3PID match for {}:{} - UID is {}", medium, address, result.getUid())
-                r = result;
-                l.countDown()
-            }
-        };
-
-        OnFailureListener failure = new OnFailureListener() {
-            @Override
-            void onFailure(@NonNull Exception e) {
-                log.info("No 3PID match for {}:{} - {}", medium, address, e.getMessage())
-                r = null;
-                l.countDown()
-            }
-        };
-
-        if (ThreePidMedium.Email.is(medium)) {
-            log.info("Performing E-mail 3PID lookup for {}", address)
-            fbAuth.getUserByEmail(address)
-                    .addOnSuccessListener(success)
-                    .addOnFailureListener(failure);
-            waitOnLatch(l);
-        } else if (ThreePidMedium.PhoneNumber.is(medium)) {
-            log.info("Performing msisdn 3PID lookup for {}", address)
-            fbAuth.getUserByPhoneNumber(address)
-                    .addOnSuccessListener(success)
-                    .addOnFailureListener(failure);
-            waitOnLatch(l);
-        } else {
-            log.info("{} is not a supported 3PID medium", medium);
-            r = null;
-        }
-
-        return Optional.ofNullable(r);
-    }
-
-    @Override
-    public Optional<SingleLookupReply> find(SingleLookupRequest request) {
-        Optional<UserRecord> urOpt = findInternal(request.getType(), request.getThreePid())
-        if (urOpt.isPresent()) {
-            return Optional.of(new SingleLookupReply(request, getMxid(urOpt.get())));
-        }
-
-        return Optional.empty();
-    }
-
-    @Override
-    public List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
-        List<ThreePidMapping> results = new ArrayList<>();
-        mappings.parallelStream().forEach(new Consumer<ThreePidMapping>() {
-            @Override
-            void accept(ThreePidMapping o) {
-                Optional<UserRecord> urOpt = findInternal(o.getMedium(), o.getValue());
-                if (urOpt.isPresent()) {
-                    ThreePidMapping result = new ThreePidMapping();
-                    result.setMedium(o.getMedium())
-                    result.setValue(o.getValue())
-                    result.setMxid(getMxid(urOpt.get()))
-                    results.add(result)
-                }
-            }
-        });
-        return results;
-    }
-
-}

src/main/groovy/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java

@@ -1,125 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.backend.ldap;
-
-import io.kamax.matrix._MatrixID;
-import io.kamax.mxisd.UserIdType;
-import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
-import io.kamax.mxisd.auth.provider.BackendAuthResult;
-import org.apache.commons.lang.StringUtils;
-import org.apache.directory.api.ldap.model.cursor.CursorException;
-import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
-import org.apache.directory.api.ldap.model.cursor.EntryCursor;
-import org.apache.directory.api.ldap.model.entry.Attribute;
-import org.apache.directory.api.ldap.model.entry.Entry;
-import org.apache.directory.api.ldap.model.exception.LdapException;
-import org.apache.directory.api.ldap.model.message.SearchScope;
-import org.apache.directory.ldap.client.api.LdapConnection;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.stereotype.Component;
-
-import java.io.IOException;
-
-@Component
-public class LdapAuthProvider extends LdapGenericBackend implements AuthenticatorProvider {
-
-    private Logger log = LoggerFactory.getLogger(LdapAuthProvider.class);
-
-    private String getUidAttribute() {
-        return getCfg().getAttribute().getUid().getValue();
-    }
-
-    @Override
-    public boolean isEnabled() {
-        return getCfg().isEnabled();
-    }
-
-    @Override
-    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
-        log.info("Performing auth for {}", mxid);
-
-        LdapConnection conn = getConn();
-        try {
-            bind(conn);
-
-            String uidType = getCfg().getAttribute().getUid().getType();
-            String userFilterValue = StringUtils.equals(LdapThreePidProvider.UID, uidType) ? mxid.getLocalPart() : mxid.getId();
-            String userFilter = "(" + getCfg().getAttribute().getUid().getValue() + "=" + userFilterValue + ")";
-            if (!StringUtils.isBlank(getCfg().getAuth().getFilter())) {
-                userFilter = "(&" + getCfg().getAuth().getFilter() + userFilter + ")";
-            }
-            EntryCursor cursor = conn.search(getCfg().getConn().getBaseDn(), userFilter, SearchScope.SUBTREE, getUidAttribute(), getCfg().getAttribute().getName());
-            try {
-                while (cursor.next()) {
-                    Entry entry = cursor.get();
-                    String dn = entry.getDn().getName();
-                    log.info("Checking possible match, DN: {}", dn);
-
-                    Attribute attribute = entry.get(getUidAttribute());
-                    if (attribute == null) {
-                        log.info("DN {}: no attribute {}, skpping", dn, getUidAttribute());
-                        continue;
-                    }
-
-                    String data = attribute.get().toString();
-                    if (data.length() < 1) {
-                        log.info("DN {}: empty attribute {}, skipping", getUidAttribute());
-                        continue;
-                    }
-
-                    log.info("Attempting authentication on LDAP for {}", dn);
-                    try {
-                        conn.bind(entry.getDn(), password);
-                    } catch (LdapException e) {
-                        log.info("Unable to bind using {} because {}", entry.getDn().getName(), e.getMessage());
-                        return BackendAuthResult.failure();
-                    }
-
-                    Attribute nameAttribute = entry.get(getCfg().getAttribute().getName());
-                    String name = nameAttribute != null ? nameAttribute.get().toString() : null;
-
-                    log.info("Authentication successful for {}", entry.getDn().getName());
-                    log.info("DN {} is a valid match", dn);
-
-                    // TODO should we canonicalize the MXID?
-                    return BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
-                }
-            } catch (CursorLdapReferralException e) {
-                log.warn("Entity for {} is only available via referral, skipping", mxid);
-            } finally {
-                cursor.close();
-            }
-
-            log.info("No match were found for {}", mxid);
-            return BackendAuthResult.failure();
-        } catch (LdapException | IOException | CursorException e) {
-            throw new RuntimeException(e);
-        } finally {
-            try {
-                conn.close();
-            } catch (IOException e) {
-                throw new RuntimeException(e);
-            }
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/backend/ldap/LdapGenericBackend.java

@@ -1,57 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.backend.ldap;
-
-import io.kamax.mxisd.config.ldap.LdapConfig;
-import org.apache.commons.lang.StringUtils;
-import org.apache.directory.api.ldap.model.exception.LdapException;
-import org.apache.directory.ldap.client.api.LdapConnection;
-import org.apache.directory.ldap.client.api.LdapNetworkConnection;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;
-
-@Component
-public class LdapGenericBackend {
-
-    private Logger log = LoggerFactory.getLogger(LdapGenericBackend.class);
-
-    @Autowired
-    private LdapConfig ldapCfg;
-
-    protected LdapConnection getConn() {
-        return new LdapNetworkConnection(ldapCfg.getConn().getHost(), ldapCfg.getConn().getPort(), ldapCfg.getConn().isTls());
-    }
-
-    protected void bind(LdapConnection conn) throws LdapException {
-        if (StringUtils.isBlank(ldapCfg.getConn().getBindDn()) && StringUtils.isBlank(ldapCfg.getConn().getBindPassword())) {
-            conn.anonymousBind();
-        } else {
-            conn.bind(ldapCfg.getConn().getBindDn(), ldapCfg.getConn().getBindPassword());
-        }
-    }
-
-    protected LdapConfig getCfg() {
-        return ldapCfg;
-    }
-
-}

src/main/groovy/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.groovy

@@ -1,169 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.backend.ldap
-
-import io.kamax.mxisd.config.MatrixConfig
-import io.kamax.mxisd.lookup.SingleLookupReply
-import io.kamax.mxisd.lookup.SingleLookupRequest
-import io.kamax.mxisd.lookup.ThreePidMapping
-import io.kamax.mxisd.lookup.provider.IThreePidProvider
-import org.apache.commons.lang.StringUtils
-import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException
-import org.apache.directory.api.ldap.model.cursor.EntryCursor
-import org.apache.directory.api.ldap.model.entry.Attribute
-import org.apache.directory.api.ldap.model.entry.Entry
-import org.apache.directory.api.ldap.model.message.SearchScope
-import org.apache.directory.ldap.client.api.LdapConnection
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.stereotype.Component
-
-@Component
-class LdapThreePidProvider extends LdapGenericBackend implements IThreePidProvider {
-
-    public static final String UID = "uid"
-    public static final String MATRIX_ID = "mxid"
-
-    private Logger log = LoggerFactory.getLogger(LdapThreePidProvider.class)
-
-    @Autowired
-    private MatrixConfig mxCfg
-
-    @Override
-    boolean isEnabled() {
-        return getCfg().isEnabled()
-    }
-
-    private String getUidAttribute() {
-        return getCfg().getAttribute().getUid().getValue();
-    }
-
-    @Override
-    boolean isLocal() {
-        return true
-    }
-
-    @Override
-    int getPriority() {
-        return 20
-    }
-
-    Optional<String> lookup(LdapConnection conn, String medium, String value) {
-        String uidAttribute = getUidAttribute()
-
-        Optional<String> queryOpt = getCfg().getIdentity().getQuery(medium)
-        if (!queryOpt.isPresent()) {
-            log.warn("{} is not a configured 3PID type for LDAP lookup", medium)
-            return Optional.empty()
-        }
-
-        String searchQuery = queryOpt.get().replaceAll("%3pid", value)
-        EntryCursor cursor = conn.search(getCfg().getConn().getBaseDn(), searchQuery, SearchScope.SUBTREE, uidAttribute)
-        try {
-            while (cursor.next()) {
-                Entry entry = cursor.get()
-                log.info("Found possible match, DN: {}", entry.getDn().getName())
-
-                Attribute attribute = entry.get(uidAttribute)
-                if (attribute == null) {
-                    log.info("DN {}: no attribute {}, skpping", entry.getDn(), getCfg().getAttribute())
-                    continue
-                }
-
-                String data = attribute.get().toString()
-                if (data.length() < 1) {
-                    log.info("DN {}: empty attribute {}, skipping", getCfg().getAttribute())
-                    continue
-                }
-
-                StringBuilder matrixId = new StringBuilder()
-                // TODO Should we turn this block into a map of functions?
-                String uidType = getCfg().getAttribute().getUid().getType()
-                if (StringUtils.equals(UID, uidType)) {
-                    matrixId.append("@").append(data).append(":").append(mxCfg.getDomain())
-                } else if (StringUtils.equals(MATRIX_ID, uidType)) {
-                    matrixId.append(data)
-                } else {
-                    log.warn("Bind was found but type {} is not supported", uidType)
-                    continue
-                }
-
-                log.info("DN {} is a valid match", entry.getDn().getName())
-                return Optional.of(matrixId.toString())
-            }
-        } catch (CursorLdapReferralException e) {
-            log.warn("3PID {} is only available via referral, skipping", value)
-        } finally {
-            cursor.close()
-        }
-
-        return Optional.empty()
-    }
-
-    @Override
-    Optional<SingleLookupReply> find(SingleLookupRequest request) {
-        log.info("Performing LDAP lookup ${request.getThreePid()} of type ${request.getType()}")
-
-        LdapConnection conn = getConn()
-        try {
-            bind(conn)
-
-            Optional<String> mxid = lookup(conn, request.getType(), request.getThreePid())
-            if (mxid.isPresent()) {
-                return Optional.of(new SingleLookupReply(request, mxid.get()));
-            }
-        } finally {
-            conn.close()
-        }
-
-        log.info("No match found")
-        return Optional.empty()
-    }
-
-    @Override
-    List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
-        log.info("Looking up {} mappings", mappings.size())
-        List<ThreePidMapping> mappingsFound = new ArrayList<>()
-
-        LdapConnection conn = getConn()
-        try {
-            bind(conn)
-
-            for (ThreePidMapping mapping : mappings) {
-                try {
-                    Optional<String> mxid = lookup(conn, mapping.getMedium(), mapping.getValue())
-                    if (mxid.isPresent()) {
-                        mapping.setMxid(mxid.get())
-                        mappingsFound.add(mapping)
-                    }
-                } catch (IllegalArgumentException e) {
-                    log.warn("{} is not a supported 3PID type for LDAP lookup", mapping.getMedium())
-                }
-            }
-        } finally {
-            conn.close()
-        }
-
-        return mappingsFound
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/DnsOverwriteEntry.java

@@ -1,67 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config;
-
-import org.apache.commons.lang.StringUtils;
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-@Configuration
-@ConfigurationProperties("dns.overwrite.homeserver")
-public class DnsOverwriteEntry {
-
-    private String name;
-    private String type;
-    private String value;
-
-    public String getName() {
-        return name;
-    }
-
-    public void setName(String name) {
-        this.name = name;
-    }
-
-    public String getType() {
-        return type;
-    }
-
-    public void setType(String type) {
-        this.type = type;
-    }
-
-    public String getValue() {
-        return value;
-    }
-
-    public void setValue(String value) {
-        this.value = value;
-    }
-
-    public String getTarget() {
-        if (StringUtils.equals("env", getType())) {
-            return System.getenv(getValue());
-        } else {
-            return getValue();
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/KeyConfig.groovy

@@ -1,50 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config
-
-import io.kamax.mxisd.exception.ConfigurationException
-import org.apache.commons.lang.StringUtils
-import org.springframework.beans.factory.InitializingBean
-import org.springframework.boot.context.properties.ConfigurationProperties
-import org.springframework.context.annotation.Configuration
-
-@Configuration
-@ConfigurationProperties(prefix = "key")
-class KeyConfig implements InitializingBean {
-
-    private String path
-
-    void setPath(String path) {
-        this.path = path
-    }
-
-    String getPath() {
-        return path
-    }
-
-    @Override
-    void afterPropertiesSet() throws Exception {
-        if (StringUtils.isBlank(getPath())) {
-            throw new ConfigurationException("key.path")
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/RecursiveLookupBridgeConfig.groovy

@@ -1,83 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config
-
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.InitializingBean
-import org.springframework.boot.context.properties.ConfigurationProperties
-import org.springframework.context.annotation.Configuration
-
-@Configuration
-@ConfigurationProperties(prefix = "lookup.recursive.bridge")
-class RecursiveLookupBridgeConfig implements InitializingBean {
-
-    private Logger log = LoggerFactory.getLogger(RecursiveLookupBridgeConfig.class)
-
-    private boolean enabled
-    private boolean recursiveOnly
-    private String server
-    private Map<String, String> mappings = new HashMap<>()
-
-    boolean getEnabled() {
-        return enabled
-    }
-
-    void setEnabled(boolean enabled) {
-        this.enabled = enabled
-    }
-
-    boolean getRecursiveOnly() {
-        return recursiveOnly
-    }
-
-    void setRecursiveOnly(boolean recursiveOnly) {
-        this.recursiveOnly = recursiveOnly
-    }
-
-    String getServer() {
-        return server
-    }
-
-    void setServer(String server) {
-        this.server = server
-    }
-
-    Map<String, String> getMappings() {
-        return mappings
-    }
-
-    void setMappings(Map<String, String> mappings) {
-        this.mappings = mappings
-    }
-
-    @Override
-    void afterPropertiesSet() throws Exception {
-        log.info("--- Bridge integration lookups config ---")
-        log.info("Enabled: {}", getEnabled())
-        if (getEnabled()) {
-            log.info("Recursive only: {}", getRecursiveOnly())
-            log.info("Fallback Server: {}", getServer())
-            log.info("Mappings: {}", mappings.size())
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/RecursiveLookupConfig.groovy

@@ -1,58 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config
-
-import org.springframework.boot.context.properties.ConfigurationProperties
-import org.springframework.context.annotation.Configuration
-
-@Configuration
-@ConfigurationProperties(prefix = "lookup.recursive")
-class RecursiveLookupConfig {
-
-    private boolean enabled
-    private List<String> allowedCidr
-    private RecursiveLookupBridgeConfig bridge
-
-    boolean isEnabled() {
-        return enabled
-    }
-
-    void setEnabled(boolean enabled) {
-        this.enabled = enabled
-    }
-
-    List<String> getAllowedCidr() {
-        return allowedCidr
-    }
-
-    void setAllowedCidr(List<String> allowedCidr) {
-        this.allowedCidr = allowedCidr
-    }
-
-    RecursiveLookupBridgeConfig getBridge() {
-        return bridge
-    }
-
-    void setBridge(RecursiveLookupBridgeConfig bridge) {
-        this.bridge = bridge
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/ServerConfig.groovy

@@ -1,95 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config
-
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.InitializingBean
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.boot.context.properties.ConfigurationProperties
-import org.springframework.context.annotation.Configuration
-
-@Configuration
-@ConfigurationProperties(prefix = "server")
-class ServerConfig implements InitializingBean {
-
-    private Logger log = LoggerFactory.getLogger(ServerConfig.class);
-
-    @Autowired
-    private MatrixConfig mxCfg;
-
-    private String name
-    private int port
-    private String publicUrl
-
-    String getName() {
-        return name
-    }
-
-    void setName(String name) {
-        this.name = name
-    }
-
-    int getPort() {
-        return port
-    }
-
-    void setPort(int port) {
-        this.port = port
-    }
-
-    String getPublicUrl() {
-        return publicUrl
-    }
-
-    void setPublicUrl(String publicUrl) {
-        this.publicUrl = publicUrl
-    }
-
-    @Override
-    void afterPropertiesSet() throws Exception {
-        log.info("--- Server config ---")
-
-        if (StringUtils.isBlank(getName())) {
-            setName(mxCfg.getDomain());
-            log.debug("server.name is empty, using matrix.domain");
-        }
-
-        if (StringUtils.isBlank(getPublicUrl())) {
-            setPublicUrl("https://${getName()}");
-            log.debug("Public URL is empty, generating from name");
-        } else {
-            setPublicUrl(StringUtils.replace(getPublicUrl(), "%SERVER_NAME%", getName()));
-        }
-
-        try {
-            new URL(getPublicUrl())
-        } catch (MalformedURLException e) {
-            log.warn("Public URL is not valid: {}", StringUtils.defaultIfBlank(e.getMessage(), "<no reason provided>"))
-        }
-
-        log.info("Name: {}", getName())
-        log.info("Port: {}", getPort())
-        log.info("Public URL: {}", getPublicUrl())
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/invite/sender/EmailSenderConfig.java

@@ -1,135 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.invite.sender;
-
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-import javax.annotation.PostConstruct;
-import java.io.File;
-
-@Configuration
-@ConfigurationProperties(prefix = "invite.sender.email")
-public class EmailSenderConfig {
-
-    private Logger log = LoggerFactory.getLogger(EmailSenderConfig.class);
-
-    private String host;
-    private int port;
-    private int tls;
-    private String login;
-    private String password;
-    private String email;
-    private String name;
-    private String template;
-
-    public String getHost() {
-        return host;
-    }
-
-    public void setHost(String host) {
-        this.host = host;
-    }
-
-    public int getPort() {
-        return port;
-    }
-
-    public void setPort(int port) {
-        this.port = port;
-    }
-
-    public int getTls() {
-        return tls;
-    }
-
-    public void setTls(int tls) {
-        this.tls = tls;
-    }
-
-    public String getLogin() {
-        return login;
-    }
-
-    public void setLogin(String login) {
-        this.login = login;
-    }
-
-    public String getPassword() {
-        return password;
-    }
-
-    public void setPassword(String password) {
-        this.password = password;
-    }
-
-    public String getEmail() {
-        return email;
-    }
-
-    public void setEmail(String email) {
-        this.email = email;
-    }
-
-    public String getName() {
-        return name;
-    }
-
-    public void setName(String name) {
-        this.name = name;
-    }
-
-    public String getTemplate() {
-        return template;
-    }
-
-    public void setTemplate(String template) {
-        this.template = template;
-    }
-
-    @PostConstruct
-    private void postConstruct() {
-        log.info("--- E-mail Invite Sender config ---");
-        log.info("Host: {}", getHost());
-        log.info("Port: {}", getPort());
-        log.info("TLS Mode: {}", getTls());
-        log.info("Login: {}", getLogin());
-        log.info("Has password: {}", StringUtils.isBlank(getPassword()));
-        log.info("E-mail: {}", getEmail());
-        if (!StringUtils.startsWith(getTemplate(), "classpath:")) {
-            if (StringUtils.isBlank(getTemplate())) {
-                log.warn("invite.sender.template is empty! Will not send invites");
-            } else {
-                File cp = new File(getTemplate()).getAbsoluteFile();
-                log.info("Template: {}", cp.getAbsolutePath());
-                if (!cp.exists() || !cp.isFile() || !cp.canRead()) {
-                    log.warn(getTemplate() + " does not exist, is not a file or cannot be read");
-                }
-            }
-        } else {
-            log.info("Template: Built-in");
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/ldap/LdapConfig.groovy

@@ -1,129 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.ldap
-
-import groovy.json.JsonOutput
-import io.kamax.mxisd.backend.ldap.LdapThreePidProvider
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.boot.context.properties.ConfigurationProperties
-import org.springframework.context.annotation.Configuration
-
-import javax.annotation.PostConstruct
-
-@Configuration
-@ConfigurationProperties(prefix = "ldap")
-class LdapConfig {
-
-    private Logger log = LoggerFactory.getLogger(LdapConfig.class)
-
-    private boolean enabled
-
-    @Autowired
-    private LdapConnectionConfig conn
-    private LdapAttributeConfig attribute
-    private LdapAuthConfig auth
-    private LdapIdentityConfig identity
-
-    boolean isEnabled() {
-        return enabled
-    }
-
-    void setEnabled(boolean enabled) {
-        this.enabled = enabled
-    }
-
-    LdapConnectionConfig getConn() {
-        return conn
-    }
-
-    void setConn(LdapConnectionConfig conn) {
-        this.conn = conn
-    }
-
-    LdapAttributeConfig getAttribute() {
-        return attribute
-    }
-
-    void setAttribute(LdapAttributeConfig attribute) {
-        this.attribute = attribute
-    }
-
-    LdapAuthConfig getAuth() {
-        return auth
-    }
-
-    void setAuth(LdapAuthConfig auth) {
-        this.auth = auth
-    }
-
-    LdapIdentityConfig getIdentity() {
-        return identity
-    }
-
-    void setIdentity(LdapIdentityConfig identity) {
-        this.identity = identity
-    }
-
-    @PostConstruct
-    void afterPropertiesSet() {
-        log.info("--- LDAP Config ---")
-        log.info("Enabled: {}", isEnabled())
-
-        if (!isEnabled()) {
-            return
-        }
-
-        if (StringUtils.isBlank(conn.getHost())) {
-            throw new IllegalStateException("LDAP Host must be configured!")
-        }
-
-        if (1 > conn.getPort() || 65535 < conn.getPort()) {
-            throw new IllegalStateException("LDAP port is not valid")
-        }
-
-        if (StringUtils.isBlank(attribute.getUid().getType())) {
-            throw new IllegalStateException("Attribute UID Type cannot be empty")
-        }
-
-
-        if (StringUtils.isBlank(attribute.getUid().getValue())) {
-            throw new IllegalStateException("Attribute UID value cannot be empty")
-        }
-
-        String uidType = attribute.getUid().getType();
-        if (!StringUtils.equals(LdapThreePidProvider.UID, uidType) && !StringUtils.equals(LdapThreePidProvider.MATRIX_ID, uidType)) {
-            throw new IllegalArgumentException("Unsupported LDAP UID type: " + uidType)
-        }
-
-        log.info("Host: {}", conn.getHost())
-        log.info("Port: {}", conn.getPort())
-        log.info("Bind DN: {}", conn.getBindDn())
-        log.info("Base DN: {}", conn.getBaseDn())
-
-        log.info("Attribute: {}", JsonOutput.toJson(attribute))
-        log.info("Auth: {}", JsonOutput.toJson(auth))
-        log.info("Identity: {}", JsonOutput.toJson(identity))
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/ldap/LdapConnectionConfig.java

@@ -1,85 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.ldap;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-@Configuration
-@ConfigurationProperties(prefix = "ldap.connection")
-public class LdapConnectionConfig {
-
-    private boolean tls;
-    private String host;
-    private int port;
-    private String bindDn;
-    private String bindPassword;
-    private String baseDn;
-
-    public boolean isTls() {
-        return tls;
-    }
-
-    public void setTls(boolean tls) {
-        this.tls = tls;
-    }
-
-    public String getHost() {
-        return host;
-    }
-
-    public void setHost(String host) {
-        this.host = host;
-    }
-
-    public int getPort() {
-        return port;
-    }
-
-    public void setPort(int port) {
-        this.port = port;
-    }
-
-    public String getBindDn() {
-        return bindDn;
-    }
-
-    public void setBindDn(String bindDn) {
-        this.bindDn = bindDn;
-    }
-
-    public String getBindPassword() {
-        return bindPassword;
-    }
-
-    public void setBindPassword(String bindPassword) {
-        this.bindPassword = bindPassword;
-    }
-
-    public String getBaseDn() {
-        return baseDn;
-    }
-
-    public void setBaseDn(String baseDn) {
-        this.baseDn = baseDn;
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/sql/SqlProviderAuthConfig.java

@@ -1,21 +0,0 @@
-package io.kamax.mxisd.config.sql;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-// Unused
-@Configuration
-@ConfigurationProperties("sql.auth")
-public class SqlProviderAuthConfig {
-
-    private boolean enabled;
-
-    public boolean isEnabled() {
-        return enabled;
-    }
-
-    public void setEnabled(boolean enabled) {
-        this.enabled = enabled;
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/sql/SqlProviderConfig.java

@@ -1,96 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.sql;
-
-import com.google.gson.Gson;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-import javax.annotation.PostConstruct;
-
-@Configuration
-@ConfigurationProperties("sql")
-public class SqlProviderConfig {
-
-    private Logger log = LoggerFactory.getLogger(SqlProviderConfig.class);
-
-    private boolean enabled;
-    private String type;
-    private String connection;
-    private SqlProviderAuthConfig auth;
-    private SqlProviderIdentityConfig identity;
-
-    public boolean isEnabled() {
-        return enabled;
-    }
-
-    public void setEnabled(boolean enabled) {
-        this.enabled = enabled;
-    }
-
-    public String getType() {
-        return type;
-    }
-
-    public void setType(String type) {
-        this.type = type;
-    }
-
-    public String getConnection() {
-        return connection;
-    }
-
-    public void setConnection(String connection) {
-        this.connection = connection;
-    }
-
-    public SqlProviderAuthConfig getAuth() {
-        return auth;
-    }
-
-    public void setAuth(SqlProviderAuthConfig auth) {
-        this.auth = auth;
-    }
-
-    public SqlProviderIdentityConfig getIdentity() {
-        return identity;
-    }
-
-    public void setIdentity(SqlProviderIdentityConfig identity) {
-        this.identity = identity;
-    }
-
-    @PostConstruct
-    private void postConstruct() {
-        log.info("--- SQL Provider config ---");
-        log.info("Enabled: {}", isEnabled());
-        if (isEnabled()) {
-            log.info("Type: {}", getType());
-            log.info("Connection: {}", getConnection());
-            log.info("Auth enabled: {}", getAuth().isEnabled());
-            log.info("Identy type: {}", getIdentity().getType());
-            log.info("Identity medium queries: {}", new Gson().toJson(getIdentity().getMedium()));
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/config/sql/SqlProviderIdentityConfig.java

@@ -1,61 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.sql;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-import java.util.HashMap;
-import java.util.Map;
-
-@Configuration
-@ConfigurationProperties("sql.identity")
-public class SqlProviderIdentityConfig {
-
-    private String type;
-    private String query;
-    private Map<String, String> medium = new HashMap<>();
-
-    public String getType() {
-        return type;
-    }
-
-    public void setType(String type) {
-        this.type = type;
-    }
-
-    public String getQuery() {
-        return query;
-    }
-
-    public void setQuery(String query) {
-        this.query = query;
-    }
-
-    public Map<String, String> getMedium() {
-        return medium;
-    }
-
-    public void setMedium(Map<String, String> medium) {
-        this.medium = medium;
-    }
-
-}

src/main/groovy/io/kamax/mxisd/controller/v1/AuthController.java

@@ -1,89 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.controller.v1;
-
-import com.google.gson.Gson;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
-import com.google.gson.JsonParser;
-import io.kamax.mxisd.auth.AuthManager;
-import io.kamax.mxisd.auth.UserAuthResult;
-import org.apache.commons.io.IOUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.http.MediaType;
-import org.springframework.web.bind.annotation.CrossOrigin;
-import org.springframework.web.bind.annotation.RequestMapping;
-import org.springframework.web.bind.annotation.RequestMethod;
-import org.springframework.web.bind.annotation.RestController;
-
-import javax.servlet.http.HttpServletRequest;
-import java.io.IOException;
-import java.nio.charset.StandardCharsets;
-
-@RestController
-@CrossOrigin
-@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-public class AuthController {
-
-    private Logger log = LoggerFactory.getLogger(AuthController.class);
-
-    private Gson gson = new Gson();
-
-    @Autowired
-    private AuthManager mgr;
-
-    @RequestMapping(value = "/_matrix-internal/identity/v1/check_credentials", method = RequestMethod.POST)
-    public String checkCredentials(HttpServletRequest req) {
-        try {
-            JsonElement el = new JsonParser().parse(IOUtils.toString(req.getInputStream(), StandardCharsets.UTF_8));
-            if (!el.isJsonObject() || !el.getAsJsonObject().has("user")) {
-                throw new IllegalArgumentException("Missing user key");
-            }
-
-            JsonObject authData = el.getAsJsonObject().get("user").getAsJsonObject();
-            if (!authData.has("id") || !authData.has("password")) {
-                throw new IllegalArgumentException("Missing id or password keys");
-            }
-
-            String id = authData.get("id").getAsString();
-            log.info("Requested to check credentials for {}", id);
-            String password = authData.get("password").getAsString();
-
-            UserAuthResult result = mgr.authenticate(id, password);
-
-            JsonObject authObj = new JsonObject();
-            authObj.addProperty("success", result.isSuccess());
-            if (result.isSuccess()) {
-                authObj.addProperty("mxid", result.getMxid());
-                authObj.addProperty("display_name", result.getDisplayName());
-            }
-            JsonObject obj = new JsonObject();
-
-            obj.add("authentication", authObj);
-            return gson.toJson(obj);
-        } catch (IOException e) {
-            throw new RuntimeException(e);
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/controller/v1/DefaultExceptionHandler.java

@@ -1,78 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.controller.v1;
-
-import com.google.gson.Gson;
-import com.google.gson.JsonObject;
-import io.kamax.mxisd.exception.BadRequestException;
-import io.kamax.mxisd.exception.MappingAlreadyExistsException;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.http.HttpStatus;
-import org.springframework.http.MediaType;
-import org.springframework.web.bind.MissingServletRequestParameterException;
-import org.springframework.web.bind.annotation.*;
-
-import javax.servlet.http.HttpServletRequest;
-
-@ControllerAdvice
-@ResponseBody
-@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-public class DefaultExceptionHandler {
-
-    private Logger log = LoggerFactory.getLogger(DefaultExceptionHandler.class);
-
-    private static Gson gson = new Gson();
-
-    static String handle(String erroCode, String error) {
-        JsonObject obj = new JsonObject();
-        obj.addProperty("errcode", erroCode);
-        obj.addProperty("error", error);
-        return gson.toJson(obj);
-    }
-
-    @ResponseStatus(HttpStatus.BAD_REQUEST)
-    @ExceptionHandler(MissingServletRequestParameterException.class)
-    public String handle(MissingServletRequestParameterException e) {
-        return handle("M_INVALID_BODY", e.getMessage());
-    }
-
-    @ResponseStatus(HttpStatus.BAD_REQUEST)
-    @ExceptionHandler(MappingAlreadyExistsException.class)
-    public String handle(MappingAlreadyExistsException e) {
-        return handle("M_ALREADY_EXISTS", e.getMessage());
-    }
-
-    @ResponseStatus(HttpStatus.BAD_REQUEST)
-    @ExceptionHandler(BadRequestException.class)
-    public String handle(BadRequestException e) {
-        return handle("M_BAD_REQUEST", e.getMessage());
-    }
-
-    @ResponseStatus(HttpStatus.INTERNAL_SERVER_ERROR)
-    @ExceptionHandler(RuntimeException.class)
-    public String handle(HttpServletRequest req, RuntimeException e) {
-        log.error("Unknown error when handling {}", req.getRequestURL(), e);
-        return handle("M_UNKNOWN", StringUtils.defaultIfBlank(e.getMessage(), "An uknown error occured. Contact the server administrator if this persists."));
-    }
-
-}

src/main/groovy/io/kamax/mxisd/controller/v1/InvitationController.groovy

@@ -1,80 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.controller.v1
-
-import com.google.gson.Gson
-import io.kamax.matrix.MatrixID
-import io.kamax.mxisd.config.ServerConfig
-import io.kamax.mxisd.controller.v1.io.ThreePidInviteReplyIO
-import io.kamax.mxisd.invitation.IThreePidInvite
-import io.kamax.mxisd.invitation.IThreePidInviteReply
-import io.kamax.mxisd.invitation.InvitationManager
-import io.kamax.mxisd.invitation.ThreePidInvite
-import io.kamax.mxisd.key.KeyManager
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.http.MediaType
-import org.springframework.web.bind.annotation.CrossOrigin
-import org.springframework.web.bind.annotation.RequestMapping
-import org.springframework.web.bind.annotation.RequestParam
-import org.springframework.web.bind.annotation.RestController
-
-import javax.servlet.http.HttpServletRequest
-
-import static org.springframework.web.bind.annotation.RequestMethod.POST
-
-@RestController
-@CrossOrigin
-@RequestMapping(path = IdentityAPIv1.BASE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-class InvitationController {
-
-    private Logger log = LoggerFactory.getLogger(InvitationController.class)
-
-    @Autowired
-    private InvitationManager mgr
-
-    @Autowired
-    private KeyManager keyMgr
-
-    @Autowired
-    private ServerConfig srvCfg
-
-    private Gson gson = new Gson()
-
-    @RequestMapping(value = "/store-invite", method = POST)
-    String store(
-            HttpServletRequest request,
-            @RequestParam String sender,
-            @RequestParam String medium,
-            @RequestParam String address,
-            @RequestParam("room_id") String roomId) {
-        Map<String, String> parameters = new HashMap<>()
-        for (String key : request.getParameterMap().keySet()) {
-            parameters.put(key, request.getParameter(key));
-        }
-        IThreePidInvite invite = new ThreePidInvite(new MatrixID(sender), medium, address, roomId, parameters)
-        IThreePidInviteReply reply = mgr.storeInvite(invite)
-
-        return gson.toJson(new ThreePidInviteReplyIO(reply, keyMgr.getPublicKeyBase64(keyMgr.getCurrentIndex()), srvCfg.getPublicUrl()))
-    }
-
-}

src/main/groovy/io/kamax/mxisd/controller/v1/KeyController.groovy

@@ -1,81 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.controller.v1
-
-import com.google.gson.Gson
-import groovy.json.JsonOutput
-import io.kamax.mxisd.controller.v1.io.KeyValidityJson
-import io.kamax.mxisd.exception.BadRequestException
-import io.kamax.mxisd.key.KeyManager
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.http.MediaType
-import org.springframework.web.bind.annotation.*
-
-import javax.servlet.http.HttpServletRequest
-
-import static org.springframework.web.bind.annotation.RequestMethod.GET
-
-@RestController
-@CrossOrigin
-@RequestMapping(path = IdentityAPIv1.BASE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-class KeyController {
-
-    private Logger log = LoggerFactory.getLogger(KeyController.class)
-
-    @Autowired
-    private KeyManager keyMgr
-
-    private Gson gson = new Gson();
-    private String validKey = gson.toJson(new KeyValidityJson(true));
-    private String invalidKey = gson.toJson(new KeyValidityJson(false));
-
-    @RequestMapping(value = "/pubkey/{keyType}:{keyId}", method = GET)
-    String getKey(@PathVariable String keyType, @PathVariable int keyId) {
-        if (!"ed25519".contentEquals(keyType)) {
-            throw new BadRequestException("Invalid algorithm: " + keyType)
-        }
-
-        log.info("Key {}:{} was requested", keyType, keyId)
-        return JsonOutput.toJson([
-                public_key: keyMgr.getPublicKeyBase64(keyId)
-        ])
-    }
-
-    @RequestMapping(value = "/pubkey/ephemeral/isvalid", method = GET)
-    String checkEphemeralKeyValidity(HttpServletRequest request) {
-        log.warn("Ephemeral key was request but no ephemeral key are generated, replying not valid")
-
-        return invalidKey
-    }
-
-    @RequestMapping(value = "/pubkey/isvalid", method = GET)
-    String checkKeyValidity(HttpServletRequest request, @RequestParam("public_key") String pubKey) {
-        log.info("Validating public key {}", pubKey)
-
-        // TODO do in manager
-        boolean valid = StringUtils.equals(pubKey, keyMgr.getPublicKeyBase64(keyMgr.getCurrentIndex()))
-        return valid ? validKey : invalidKey
-    }
-
-}

src/main/groovy/io/kamax/mxisd/controller/v1/MappingController.groovy

@@ -1,121 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.controller.v1
-
-import com.google.gson.Gson
-import com.google.gson.JsonObject
-import groovy.json.JsonOutput
-import groovy.json.JsonSlurper
-import io.kamax.mxisd.controller.v1.io.SingeLookupReplyJson
-import io.kamax.mxisd.lookup.*
-import io.kamax.mxisd.lookup.strategy.LookupStrategy
-import io.kamax.mxisd.signature.SignatureManager
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.http.MediaType
-import org.springframework.web.bind.annotation.CrossOrigin
-import org.springframework.web.bind.annotation.RequestMapping
-import org.springframework.web.bind.annotation.RequestParam
-import org.springframework.web.bind.annotation.RestController
-
-import javax.servlet.http.HttpServletRequest
-
-import static org.springframework.web.bind.annotation.RequestMethod.GET
-import static org.springframework.web.bind.annotation.RequestMethod.POST
-
-@RestController
-@CrossOrigin
-@RequestMapping(path = IdentityAPIv1.BASE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-class MappingController {
-
-    private Logger log = LoggerFactory.getLogger(MappingController.class)
-    private JsonSlurper json = new JsonSlurper()
-    private Gson gson = new Gson()
-
-    @Autowired
-    private LookupStrategy strategy
-
-    @Autowired
-    private SignatureManager signMgr
-
-    private void setRequesterInfo(ALookupRequest lookupReq, HttpServletRequest req) {
-        lookupReq.setRequester(req.getRemoteAddr())
-        String xff = req.getHeader("X-FORWARDED-FOR")
-        lookupReq.setRecursive(StringUtils.isNotBlank(xff))
-        if (lookupReq.isRecursive()) {
-            lookupReq.setRecurseHosts(Arrays.asList(xff.split(",")))
-        }
-
-        lookupReq.setUserAgent(req.getHeader("USER-AGENT"))
-    }
-
-    @RequestMapping(value = "/lookup", method = GET)
-    String lookup(HttpServletRequest request, @RequestParam String medium, @RequestParam String address) {
-        SingleLookupRequest lookupRequest = new SingleLookupRequest()
-        setRequesterInfo(lookupRequest, request)
-        lookupRequest.setType(medium)
-        lookupRequest.setThreePid(address)
-
-        log.info("Got single lookup request from {} with client {} - Is recursive? {}", lookupRequest.getRequester(), lookupRequest.getUserAgent(), lookupRequest.isRecursive())
-
-        Optional<SingleLookupReply> lookupOpt = strategy.find(lookupRequest)
-        if (!lookupOpt.isPresent()) {
-            log.info("No mapping was found, return empty JSON object")
-            return JsonOutput.toJson([])
-        }
-
-        SingleLookupReply lookup = lookupOpt.get()
-        if (lookup.isSigned()) {
-            log.info("Lookup is already signed, sending as-is")
-            return lookup.getBody();
-        } else {
-            log.info("Lookup is not signed, signing")
-            JsonObject obj = new Gson().toJsonTree(new SingeLookupReplyJson(lookup)).getAsJsonObject()
-            obj.add("signatures", signMgr.signMessageGson(gson.toJson(obj)))
-
-            return gson.toJson(obj)
-        }
-    }
-
-    @RequestMapping(value = "/bulk_lookup", method = POST)
-    String bulkLookup(HttpServletRequest request) {
-        BulkLookupRequest lookupRequest = new BulkLookupRequest()
-        setRequesterInfo(lookupRequest, request)
-        log.info("Got single lookup request from {} with client {} - Is recursive? {}", lookupRequest.getRequester(), lookupRequest.getUserAgent(), lookupRequest.isRecursive())
-
-        ClientBulkLookupRequest input = (ClientBulkLookupRequest) json.parseText(request.getInputStream().getText())
-        List<ThreePidMapping> mappings = new ArrayList<>()
-        for (List<String> mappingRaw : input.getThreepids()) {
-            ThreePidMapping mapping = new ThreePidMapping()
-            mapping.setMedium(mappingRaw.get(0))
-            mapping.setValue(mappingRaw.get(1))
-            mappings.add(mapping)
-        }
-        lookupRequest.setMappings(mappings)
-
-        ClientBulkLookupAnswer answer = new ClientBulkLookupAnswer()
-        answer.addAll(strategy.find(lookupRequest))
-        return JsonOutput.toJson(answer)
-    }
-
-}

src/main/groovy/io/kamax/mxisd/controller/v1/SessionController.groovy

@@ -1,162 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.controller.v1
-
-import com.google.gson.Gson
-import com.google.gson.JsonObject
-import io.kamax.mxisd.controller.v1.io.SessionEmailTokenRequestJson
-import io.kamax.mxisd.controller.v1.io.SessionPhoneTokenRequestJson
-import io.kamax.mxisd.exception.BadRequestException
-import io.kamax.mxisd.invitation.InvitationManager
-import io.kamax.mxisd.lookup.ThreePidValidation
-import io.kamax.mxisd.mapping.MappingManager
-import org.apache.commons.io.IOUtils
-import org.apache.commons.lang.StringUtils
-import org.apache.http.HttpStatus
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.http.MediaType
-import org.springframework.web.bind.annotation.*
-
-import javax.servlet.http.HttpServletRequest
-import javax.servlet.http.HttpServletResponse
-import java.nio.charset.StandardCharsets
-
-@RestController
-@CrossOrigin
-@RequestMapping(path = IdentityAPIv1.BASE, produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-class SessionController {
-
-    @Autowired
-    private MappingManager mgr
-
-    @Autowired
-    private InvitationManager invMgr;
-
-    private Gson gson = new Gson()
-
-    private Logger log = LoggerFactory.getLogger(SessionController.class)
-
-    private <T> T fromJson(HttpServletRequest req, Class<T> obj) {
-        gson.fromJson(new InputStreamReader(req.getInputStream(), StandardCharsets.UTF_8), obj)
-    }
-
-    @RequestMapping(value = "/validate/{medium}/requestToken")
-    String init(HttpServletRequest request, HttpServletResponse response, @PathVariable String medium) {
-        log.info("Requested: {}", request.getRequestURL(), request.getQueryString())
-
-        if (StringUtils.equals("email", medium)) {
-            SessionEmailTokenRequestJson req = fromJson(request, SessionEmailTokenRequestJson.class)
-            return gson.toJson(new Sid(mgr.create(req)))
-        }
-
-        if (StringUtils.equals("msisdn", medium)) {
-            SessionPhoneTokenRequestJson req = fromJson(request, SessionPhoneTokenRequestJson.class)
-            return gson.toJson(new Sid(mgr.create(req)))
-        }
-
-        JsonObject obj = new JsonObject();
-        obj.addProperty("errcode", "M_INVALID_3PID_TYPE")
-        obj.addProperty("error", medium + " is not supported as a 3PID type")
-        response.setStatus(HttpStatus.SC_BAD_REQUEST)
-        return gson.toJson(obj)
-    }
-
-    @RequestMapping(value = "/validate/{medium}/submitToken")
-    String validate(HttpServletRequest request,
-                    @RequestParam String sid,
-                    @RequestParam("client_secret") String secret, @RequestParam String token) {
-        log.info("Requested: {}?{}", request.getRequestURL(), request.getQueryString())
-
-        mgr.validate(sid, secret, token)
-
-        return "{}"
-    }
-
-    @RequestMapping(value = "/3pid/getValidated3pid")
-    String check(HttpServletRequest request, HttpServletResponse response,
-                 @RequestParam String sid, @RequestParam("client_secret") String secret) {
-        log.info("Requested: {}?{}", request.getRequestURL(), request.getQueryString())
-
-        Optional<ThreePidValidation> result = mgr.getValidated(sid, secret)
-        if (result.isPresent()) {
-            log.info("requested session was validated")
-            ThreePidValidation pid = result.get()
-
-            JsonObject obj = new JsonObject()
-            obj.addProperty("medium", pid.getMedium())
-            obj.addProperty("address", pid.getAddress())
-            obj.addProperty("validated_at", pid.getValidation().toEpochMilli())
-
-            return gson.toJson(obj);
-        } else {
-            log.info("requested session was not validated")
-
-            JsonObject obj = new JsonObject()
-            obj.addProperty("errcode", "M_SESSION_NOT_VALIDATED")
-            obj.addProperty("error", "sid, secret or session not valid")
-            response.setStatus(HttpStatus.SC_BAD_REQUEST)
-            return gson.toJson(obj)
-        }
-    }
-
-    @RequestMapping(value = "/3pid/bind")
-    String bind(HttpServletRequest request, HttpServletResponse response,
-                @RequestParam String sid, @RequestParam("client_secret") String secret, @RequestParam String mxid) {
-        String data = IOUtils.toString(request.getReader())
-        log.info("Requested: {}", request.getRequestURL(), request.getQueryString())
-        try {
-            mgr.bind(sid, secret, mxid)
-            return "{}"
-        } catch (BadRequestException e) {
-            log.info("requested session was not validated")
-
-            JsonObject obj = new JsonObject()
-            obj.addProperty("errcode", "M_SESSION_NOT_VALIDATED")
-            obj.addProperty("error", e.getMessage())
-            response.setStatus(HttpStatus.SC_BAD_REQUEST)
-            return gson.toJson(obj)
-        } finally {
-            // If a user registers, there is no standard login event. Instead, this is the only way to trigger
-            // resolution at an appropriate time. Meh at synapse/Riot!
-            invMgr.lookupMappingsForInvites()
-        }
-    }
-
-    private class Sid {
-
-        private String sid;
-
-        public Sid(String sid) {
-            setSid(sid);
-        }
-
-        String getSid() {
-            return sid
-        }
-
-        void setSid(String sid) {
-            this.sid = sid
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/invitation/sender/EmailInviteSender.java

@@ -1,138 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.invitation.sender;
-
-import com.sun.mail.smtp.SMTPTransport;
-import io.kamax.matrix.ThreePidMedium;
-import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.invite.sender.EmailSenderConfig;
-import io.kamax.mxisd.exception.ConfigurationException;
-import io.kamax.mxisd.invitation.IThreePidInviteReply;
-import org.apache.commons.io.IOUtils;
-import org.apache.commons.lang.StringUtils;
-import org.apache.commons.lang.WordUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.context.ApplicationContext;
-import org.springframework.stereotype.Component;
-
-import javax.annotation.PostConstruct;
-import javax.mail.Message;
-import javax.mail.MessagingException;
-import javax.mail.Session;
-import javax.mail.internet.InternetAddress;
-import javax.mail.internet.MimeMessage;
-import java.io.FileInputStream;
-import java.io.IOException;
-import java.io.UnsupportedEncodingException;
-import java.nio.charset.StandardCharsets;
-import java.util.Date;
-
-@Component
-public class EmailInviteSender implements IInviteSender {
-
-    private Logger log = LoggerFactory.getLogger(EmailInviteSender.class);
-
-    @Autowired
-    private EmailSenderConfig cfg;
-
-    @Autowired
-    private MatrixConfig mxCfg;
-
-    @Autowired
-    private ApplicationContext app;
-
-    private Session session;
-    private InternetAddress sender;
-
-    @PostConstruct
-    private void postConstruct() {
-        try {
-            session = Session.getInstance(System.getProperties());
-            sender = new InternetAddress(cfg.getEmail(), cfg.getName());
-        } catch (UnsupportedEncodingException e) {
-            // What are we supposed to do with this?!
-            throw new ConfigurationException(e);
-        }
-    }
-
-    @Override
-    public String getMedium() {
-        return ThreePidMedium.Email.getId();
-    }
-
-    @Override
-    public void send(IThreePidInviteReply invite) {
-        if (!ThreePidMedium.Email.is(invite.getInvite().getMedium())) {
-            throw new IllegalArgumentException(invite.getInvite().getMedium() + " is not a supported 3PID type");
-        }
-
-        try {
-            String domainPretty = WordUtils.capitalizeFully(mxCfg.getDomain());
-            String senderName = invite.getInvite().getProperties().getOrDefault("sender_display_name", "");
-            String senderNameOrId = StringUtils.defaultIfBlank(senderName, invite.getInvite().getSender().getId());
-            String roomName = invite.getInvite().getProperties().getOrDefault("room_name", "");
-            String roomNameOrId = StringUtils.defaultIfBlank(roomName, invite.getInvite().getRoomId());
-
-            String templateBody = IOUtils.toString(
-                    StringUtils.startsWith(cfg.getTemplate(), "classpath:") ?
-                            app.getResource(cfg.getTemplate()).getInputStream() : new FileInputStream(cfg.getTemplate()),
-                    StandardCharsets.UTF_8);
-            templateBody = templateBody.replace("%DOMAIN%", mxCfg.getDomain());
-            templateBody = templateBody.replace("%DOMAIN_PRETTY%", domainPretty);
-            templateBody = templateBody.replace("%FROM_EMAIL%", cfg.getEmail());
-            templateBody = templateBody.replace("%FROM_NAME%", cfg.getName());
-            templateBody = templateBody.replace("%SENDER_ID%", invite.getInvite().getSender().getId());
-            templateBody = templateBody.replace("%SENDER_NAME%", senderName);
-            templateBody = templateBody.replace("%SENDER_NAME_OR_ID%", senderNameOrId);
-            templateBody = templateBody.replace("%INVITE_MEDIUM%", invite.getInvite().getMedium());
-            templateBody = templateBody.replace("%INVITE_ADDRESS%", invite.getInvite().getAddress());
-            templateBody = templateBody.replace("%ROOM_ID%", invite.getInvite().getRoomId());
-            templateBody = templateBody.replace("%ROOM_NAME%", roomName);
-            templateBody = templateBody.replace("%ROOM_NAME_OR_ID%", roomNameOrId);
-
-            MimeMessage msg = new MimeMessage(session, IOUtils.toInputStream(templateBody, StandardCharsets.UTF_8));
-            msg.setHeader("X-Mailer", "mxisd"); // TODO set version
-            msg.setSentDate(new Date());
-            msg.setFrom(sender);
-            msg.setRecipients(Message.RecipientType.TO, invite.getInvite().getAddress());
-            msg.saveChanges();
-
-            log.info("Sending invite to {} via SMTP using {}:{}", invite.getInvite().getAddress(), cfg.getHost(), cfg.getPort());
-            SMTPTransport transport = (SMTPTransport) session.getTransport("smtp");
-            transport.setStartTLS(cfg.getTls() > 0);
-            transport.setRequireStartTLS(cfg.getTls() > 1);
-
-            log.info("Connecting to {}:{}", cfg.getHost(), cfg.getPort());
-            transport.connect(cfg.getHost(), cfg.getPort(), cfg.getLogin(), cfg.getPassword());
-            try {
-                transport.sendMessage(msg, InternetAddress.parse(invite.getInvite().getAddress()));
-                log.info("Invite to {} was sent", invite.getInvite().getAddress());
-            } finally {
-                transport.close();
-            }
-        } catch (IOException | MessagingException e) {
-            throw new RuntimeException("Unable to send e-mail invite to " + invite.getInvite().getAddress(), e);
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/key/KeyManager.groovy

@@ -1,106 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.key
-
-import io.kamax.mxisd.config.KeyConfig
-import net.i2p.crypto.eddsa.EdDSAEngine
-import net.i2p.crypto.eddsa.EdDSAPrivateKey
-import net.i2p.crypto.eddsa.EdDSAPublicKey
-import net.i2p.crypto.eddsa.KeyPairGenerator
-import net.i2p.crypto.eddsa.spec.EdDSANamedCurveTable
-import net.i2p.crypto.eddsa.spec.EdDSAParameterSpec
-import net.i2p.crypto.eddsa.spec.EdDSAPrivateKeySpec
-import net.i2p.crypto.eddsa.spec.EdDSAPublicKeySpec
-import org.apache.commons.io.FileUtils
-import org.springframework.beans.factory.InitializingBean
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.stereotype.Component
-
-import java.nio.charset.StandardCharsets
-import java.nio.file.Files
-import java.nio.file.Path
-import java.nio.file.Paths
-import java.security.KeyPair
-import java.security.MessageDigest
-import java.security.PrivateKey
-
-@Component
-class KeyManager implements InitializingBean {
-
-    @Autowired
-    private KeyConfig keyCfg
-
-    private EdDSAParameterSpec keySpecs
-    private EdDSAEngine signEngine
-    private List<KeyPair> keys
-
-    @Override
-    void afterPropertiesSet() throws Exception {
-        keySpecs = EdDSANamedCurveTable.getByName(EdDSANamedCurveTable.CURVE_ED25519_SHA512)
-        signEngine = new EdDSAEngine(MessageDigest.getInstance(keySpecs.getHashAlgorithm()))
-        keys = new ArrayList<>()
-
-        Path privKey = Paths.get(keyCfg.getPath())
-
-        if (!Files.exists(privKey)) {
-            KeyPair pair = (new KeyPairGenerator()).generateKeyPair()
-            String keyEncoded = Base64.getEncoder().encodeToString(pair.getPrivate().getEncoded())
-            FileUtils.writeStringToFile(privKey.toFile(), keyEncoded, StandardCharsets.ISO_8859_1)
-            keys.add(pair)
-        } else {
-            if (Files.isDirectory(privKey)) {
-                throw new RuntimeException("Invalid path for private key: ${privKey.toString()}")
-            }
-
-            if (Files.isReadable(privKey)) {
-                byte[] seed = Base64.getDecoder().decode(FileUtils.readFileToString(privKey.toFile(), StandardCharsets.ISO_8859_1))
-                EdDSAPrivateKeySpec privKeySpec = new EdDSAPrivateKeySpec(seed, keySpecs)
-                EdDSAPublicKeySpec pubKeySpec = new EdDSAPublicKeySpec(privKeySpec.getA(), keySpecs)
-                keys.add(new KeyPair(new EdDSAPublicKey(pubKeySpec), new EdDSAPrivateKey(privKeySpec)))
-            }
-        }
-    }
-
-    int getCurrentIndex() {
-        return 0
-    }
-
-    KeyPair getKeys(int index) {
-        return keys.get(index)
-    }
-
-    PrivateKey getPrivateKey(int index) {
-        return getKeys(index).getPrivate()
-    }
-
-    EdDSAPublicKey getPublicKey(int index) {
-        return (EdDSAPublicKey) getKeys(index).getPublic()
-    }
-
-    EdDSAParameterSpec getSpecs() {
-        return keySpecs
-    }
-
-    String getPublicKeyBase64(int index) {
-        return Base64.getEncoder().encodeToString(getPublicKey(index).getAbyte())
-    }
-
-}

src/main/groovy/io/kamax/mxisd/lookup/provider/DnsLookupProvider.groovy

@@ -1,231 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.lookup.provider
-
-import io.kamax.mxisd.config.MatrixConfig
-import io.kamax.mxisd.lookup.SingleLookupReply
-import io.kamax.mxisd.lookup.SingleLookupRequest
-import io.kamax.mxisd.lookup.ThreePidMapping
-import io.kamax.mxisd.lookup.fetcher.IRemoteIdentityServerFetcher
-import org.apache.commons.lang.StringUtils
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.stereotype.Component
-import org.xbill.DNS.Lookup
-import org.xbill.DNS.SRVRecord
-import org.xbill.DNS.Type
-
-import java.util.concurrent.ForkJoinPool
-import java.util.concurrent.RecursiveTask
-import java.util.function.Function
-
-@Component
-class DnsLookupProvider implements IThreePidProvider {
-
-    private Logger log = LoggerFactory.getLogger(DnsLookupProvider.class)
-
-    @Autowired
-    private MatrixConfig mxCfg
-
-    @Autowired
-    private IRemoteIdentityServerFetcher fetcher
-
-    @Override
-    boolean isEnabled() {
-        return true
-    }
-
-    @Override
-    boolean isLocal() {
-        return false
-    }
-
-    @Override
-    int getPriority() {
-        return 10
-    }
-
-    String getSrvRecordName(String domain) {
-        return "_matrix-identity._tcp." + domain
-    }
-
-    Optional<String> getDomain(String email) {
-        int atIndex = email.lastIndexOf("@")
-        if (atIndex == -1) {
-            return Optional.empty()
-        }
-
-        return Optional.of(email.substring(atIndex + 1))
-    }
-
-    // TODO use caching mechanism
-    Optional<String> findIdentityServerForDomain(String domain) {
-        if (StringUtils.equals(mxCfg.getDomain(), domain)) {
-            log.info("We are authoritative for {}, no remote lookup", domain)
-            return Optional.empty()
-        }
-
-        log.info("Performing SRV lookup")
-        String lookupDns = getSrvRecordName(domain)
-        log.info("Lookup name: {}", lookupDns)
-
-        SRVRecord[] records = (SRVRecord[]) new Lookup(lookupDns, Type.SRV).run()
-        if (records != null) {
-            Arrays.sort(records, new Comparator<SRVRecord>() {
-
-                @Override
-                int compare(SRVRecord o1, SRVRecord o2) {
-                    return Integer.compare(o1.getPriority(), o2.getPriority())
-                }
-
-            })
-
-            for (SRVRecord record : records) {
-                log.info("Found SRV record: {}", record.toString())
-                String baseUrl = "https://${record.getTarget().toString(true)}:${record.getPort()}"
-                if (fetcher.isUsable(baseUrl)) {
-                    log.info("Found Identity Server for domain {} at {}", domain, baseUrl)
-                    return Optional.of(baseUrl)
-                } else {
-                    log.info("{} is not a usable Identity Server", baseUrl)
-                }
-            }
-        } else {
-            log.info("No SRV record for {}", lookupDns)
-        }
-
-        log.info("Performing basic lookup using domain name {}", domain)
-        String baseUrl = "https://" + domain
-        if (fetcher.isUsable(baseUrl)) {
-            log.info("Found Identity Server for domain {} at {}", domain, baseUrl)
-            return Optional.of(baseUrl)
-        } else {
-            log.info("{} is not a usable Identity Server", baseUrl)
-            return Optional.empty()
-        }
-    }
-
-    @Override
-    Optional<SingleLookupReply> find(SingleLookupRequest request) {
-        if (!StringUtils.equals("email", request.getType())) { // TODO use enum
-            log.info("Skipping unsupported type {} for {}", request.getType(), request.getThreePid())
-            return Optional.empty()
-        }
-
-        log.info("Performing DNS lookup for {}", request.getThreePid())
-
-        String domain = request.getThreePid().substring(request.getThreePid().lastIndexOf("@") + 1)
-        log.info("Domain name for {}: {}", request.getThreePid(), domain)
-        Optional<String> baseUrl = findIdentityServerForDomain(domain)
-
-        if (baseUrl.isPresent()) {
-            return fetcher.find(baseUrl.get(), request)
-        }
-
-        return Optional.empty()
-    }
-
-    @Override
-    List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
-        Map<String, List<ThreePidMapping>> domains = new HashMap<>()
-
-        for (ThreePidMapping mapping : mappings) {
-            if (!StringUtils.equals("email", mapping.getMedium())) {
-                log.info("Skipping unsupported type {} for {}", mapping.getMedium(), mapping.getValue())
-                continue
-            }
-
-            Optional<String> domainOpt = getDomain(mapping.getValue())
-            if (!domainOpt.isPresent()) {
-                log.warn("No domain for 3PID {}", mapping.getValue())
-                continue
-            }
-
-            String domain = domainOpt.get()
-            List<ThreePidMapping> domainMappings = domains.computeIfAbsent(domain, new Function<String, List<ThreePidMapping>>() {
-
-                @Override
-                List<ThreePidMapping> apply(String s) {
-                    return new ArrayList<>()
-                }
-
-            })
-            domainMappings.add(mapping)
-        }
-
-        log.info("Looking mappings across {} domains", domains.keySet().size())
-        ForkJoinPool pool = new ForkJoinPool()
-        RecursiveTask<List<ThreePidMapping>> task = new RecursiveTask<List<ThreePidMapping>>() {
-
-            @Override
-            protected List<ThreePidMapping> compute() {
-                List<ThreePidMapping> mappingsFound = new ArrayList<>()
-                List<DomainBulkLookupTask> tasks = new ArrayList<>()
-
-                for (String domain : domains.keySet()) {
-                    DomainBulkLookupTask domainTask = new DomainBulkLookupTask(domain, domains.get(domain))
-                    domainTask.fork()
-                    tasks.add(domainTask)
-                }
-
-                for (DomainBulkLookupTask task : tasks) {
-                    mappingsFound.addAll(task.join())
-                }
-
-                return mappingsFound
-            }
-        }
-        pool.submit(task)
-        pool.shutdown()
-
-        List<ThreePidMapping> mappingsFound = task.join()
-        log.info("Found {} mappings overall", mappingsFound.size())
-        return mappingsFound
-    }
-
-    private class DomainBulkLookupTask extends RecursiveTask<List<ThreePidMapping>> {
-
-        private String domain
-        private List<ThreePidMapping> mappings
-
-        DomainBulkLookupTask(String domain, List<ThreePidMapping> mappings) {
-            this.domain = domain
-            this.mappings = mappings
-        }
-
-        @Override
-        protected List<ThreePidMapping> compute() {
-            List<ThreePidMapping> domainMappings = new ArrayList<>()
-
-            Optional<String> baseUrl = findIdentityServerForDomain(domain)
-            if (!baseUrl.isPresent()) {
-                log.info("No usable Identity server for domain {}", domain)
-            } else {
-                domainMappings.addAll(fetcher.find(baseUrl.get(), mappings))
-                log.info("Found {} mappings in domain {}", domainMappings.size(), domain)
-            }
-
-            return domainMappings
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/lookup/provider/ForwarderProvider.groovy

@@ -1,88 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.lookup.provider
-
-import io.kamax.mxisd.config.ForwardConfig
-import io.kamax.mxisd.lookup.SingleLookupReply
-import io.kamax.mxisd.lookup.SingleLookupRequest
-import io.kamax.mxisd.lookup.ThreePidMapping
-import io.kamax.mxisd.lookup.fetcher.IRemoteIdentityServerFetcher
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.stereotype.Component
-
-@Component
-class ForwarderProvider implements IThreePidProvider {
-
-    private Logger log = LoggerFactory.getLogger(ForwarderProvider.class)
-
-    @Autowired
-    private ForwardConfig cfg
-
-    @Autowired
-    private IRemoteIdentityServerFetcher fetcher
-
-    @Override
-    boolean isEnabled() {
-        return true
-    }
-
-    @Override
-    boolean isLocal() {
-        return false
-    }
-
-    @Override
-    int getPriority() {
-        return 0
-    }
-
-    @Override
-    Optional<SingleLookupReply> find(SingleLookupRequest request) {
-        for (String root : cfg.getServers()) {
-            Optional<SingleLookupReply> answer = fetcher.find(root, request)
-            if (answer.isPresent()) {
-                return answer
-            }
-        }
-
-        return Optional.empty()
-    }
-
-    @Override
-    List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
-        List<ThreePidMapping> mappingsToDo = new ArrayList<>(mappings)
-        List<ThreePidMapping> mappingsFoundGlobal = new ArrayList<>()
-
-        for (String root : cfg.getServers()) {
-            log.info("{} mappings remaining: {}", mappingsToDo.size(), mappingsToDo)
-            log.info("Querying {}", root)
-            List<ThreePidMapping> mappingsFound = fetcher.find(root, mappingsToDo)
-            log.info("{} returned {} mappings", root, mappingsFound.size())
-            mappingsFoundGlobal.addAll(mappingsFound)
-            mappingsToDo.removeAll(mappingsFound)
-        }
-
-        return mappingsFoundGlobal
-    }
-
-}

src/main/groovy/io/kamax/mxisd/lookup/provider/RemoteIdentityServerFetcher.groovy

@@ -1,157 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.lookup.provider
-
-import groovy.json.JsonException
-import groovy.json.JsonOutput
-import groovy.json.JsonSlurper
-import io.kamax.mxisd.controller.v1.ClientBulkLookupRequest
-import io.kamax.mxisd.lookup.SingleLookupReply
-import io.kamax.mxisd.lookup.SingleLookupRequest
-import io.kamax.mxisd.lookup.ThreePidMapping
-import io.kamax.mxisd.lookup.fetcher.IRemoteIdentityServerFetcher
-import org.apache.http.HttpEntity
-import org.apache.http.HttpResponse
-import org.apache.http.client.HttpClient
-import org.apache.http.client.entity.EntityBuilder
-import org.apache.http.client.methods.HttpPost
-import org.apache.http.entity.ContentType
-import org.apache.http.impl.client.HttpClients
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.context.annotation.Lazy
-import org.springframework.context.annotation.Scope
-import org.springframework.stereotype.Component
-
-@Component
-@Scope("prototype")
-@Lazy
-public class RemoteIdentityServerFetcher implements IRemoteIdentityServerFetcher {
-
-    public static final String THREEPID_TEST_MEDIUM = "email"
-    public static final String THREEPID_TEST_ADDRESS = "john.doe@example.org"
-
-    private Logger log = LoggerFactory.getLogger(RemoteIdentityServerFetcher.class)
-
-    private JsonSlurper json = new JsonSlurper()
-
-    @Override
-    boolean isUsable(String remote) {
-        try {
-            HttpURLConnection rootSrvConn = (HttpURLConnection) new URL(
-                    "${remote}/_matrix/identity/api/v1/lookup?medium=${THREEPID_TEST_MEDIUM}&address=${THREEPID_TEST_ADDRESS}"
-            ).openConnection()
-            // TODO turn this into a configuration property
-            rootSrvConn.setConnectTimeout(2000)
-
-            if (rootSrvConn.getResponseCode() != 200) {
-                return false
-            }
-
-            def output = json.parseText(rootSrvConn.getInputStream().getText())
-            if (output['address']) {
-                return false
-            }
-
-            return true
-        } catch (IOException | JsonException e) {
-            log.info("{} is not a usable Identity Server: {}", remote, e.getMessage())
-            return false
-        }
-    }
-
-    @Override
-    Optional<SingleLookupReply> find(String remote, SingleLookupRequest request) {
-        log.info("Looking up {} 3PID {} using {}", request.getType(), request.getThreePid(), remote)
-
-        HttpURLConnection rootSrvConn = (HttpURLConnection) new URL(
-                "${remote}/_matrix/identity/api/v1/lookup?medium=${request.getType()}&address=${request.getThreePid()}"
-        ).openConnection()
-
-        try {
-            String outputRaw = rootSrvConn.getInputStream().getText()
-            def output = json.parseText(outputRaw)
-            if (output['address']) {
-                log.info("Found 3PID mapping: {}", output)
-
-                return Optional.of(SingleLookupReply.fromRecursive(request, outputRaw))
-            }
-
-            log.info("Empty 3PID mapping from {}", remote)
-            return Optional.empty()
-        } catch (IOException e) {
-            log.warn("Error looking up 3PID mapping {}: {}", request.getThreePid(), e.getMessage())
-            return Optional.empty()
-        } catch (JsonException e) {
-            log.warn("Invalid JSON answer from {}", remote)
-            return Optional.empty()
-        }
-    }
-
-    @Override
-    List<ThreePidMapping> find(String remote, List<ThreePidMapping> mappings) {
-        List<ThreePidMapping> mappingsFound = new ArrayList<>()
-
-        ClientBulkLookupRequest mappingRequest = new ClientBulkLookupRequest()
-        mappingRequest.setMappings(mappings)
-
-        String url = "${remote}/_matrix/identity/api/v1/bulk_lookup"
-        HttpClient client = HttpClients.createDefault()
-        try {
-            HttpPost request = new HttpPost(url)
-            request.setEntity(
-                    EntityBuilder.create()
-                            .setText(JsonOutput.toJson(mappingRequest))
-                            .setContentType(ContentType.APPLICATION_JSON)
-                            .build()
-            )
-
-            HttpResponse response = client.execute(request)
-            try {
-                if (response.getStatusLine().getStatusCode() != 200) {
-                    log.info("Could not perform lookup at {} due to HTTP return code: {}", url, response.getStatusLine().getStatusCode())
-                    return mappingsFound
-                }
-
-                HttpEntity entity = response.getEntity()
-                if (entity != null) {
-                    ClientBulkLookupRequest input = (ClientBulkLookupRequest) json.parseText(entity.getContent().getText())
-                    for (List<String> mappingRaw : input.getThreepids()) {
-                        ThreePidMapping mapping = new ThreePidMapping()
-                        mapping.setMedium(mappingRaw.get(0))
-                        mapping.setValue(mappingRaw.get(1))
-                        mapping.setMxid(mappingRaw.get(2))
-                        mappingsFound.add(mapping)
-                    }
-                } else {
-                    log.info("HTTP response from {} was empty", remote)
-                }
-
-                return mappingsFound
-            } finally {
-                response.close()
-            }
-        } finally {
-            client.close()
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/lookup/strategy/RecursivePriorityLookupStrategy.groovy

@@ -1,183 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.lookup.strategy
-
-import edazdarevic.commons.net.CIDRUtils
-import io.kamax.mxisd.config.RecursiveLookupConfig
-import io.kamax.mxisd.lookup.*
-import io.kamax.mxisd.lookup.fetcher.IBridgeFetcher
-import io.kamax.mxisd.lookup.provider.IThreePidProvider
-import org.slf4j.Logger
-import org.slf4j.LoggerFactory
-import org.springframework.beans.factory.InitializingBean
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.stereotype.Component
-
-import java.util.function.Predicate
-import java.util.stream.Collectors
-
-@Component
-class RecursivePriorityLookupStrategy implements LookupStrategy, InitializingBean {
-
-    private Logger log = LoggerFactory.getLogger(RecursivePriorityLookupStrategy.class)
-
-    @Autowired
-    private RecursiveLookupConfig recursiveCfg
-
-    @Autowired
-    private List<IThreePidProvider> providers
-
-    @Autowired
-    private IBridgeFetcher bridge
-
-    private List<CIDRUtils> allowedCidr = new ArrayList<>()
-
-    @Override
-    void afterPropertiesSet() throws Exception {
-        log.info("Found ${providers.size()} providers")
-
-        providers.sort(new Comparator<IThreePidProvider>() {
-
-            @Override
-            int compare(IThreePidProvider o1, IThreePidProvider o2) {
-                return Integer.compare(o2.getPriority(), o1.getPriority())
-            }
-
-        })
-
-        log.info("Recursive lookup enabled: {}", recursiveCfg.isEnabled())
-        for (String cidr : recursiveCfg.getAllowedCidr()) {
-            log.info("{} is allowed for recursion", cidr)
-            allowedCidr.add(new CIDRUtils(cidr))
-        }
-    }
-
-    boolean isAllowedForRecursive(String source) {
-        boolean canRecurse = false
-
-        if (recursiveCfg.isEnabled()) {
-            log.debug("Checking {} CIDRs for recursion", allowedCidr.size())
-            for (CIDRUtils cidr : allowedCidr) {
-                if (cidr.isInRange(source)) {
-                    log.debug("{} is in range {}, allowing recursion", source, cidr.getNetworkAddress())
-                    canRecurse = true
-                    break
-                } else {
-                    log.debug("{} is not in range {}", source, cidr.getNetworkAddress())
-                }
-            }
-        }
-
-        return canRecurse
-    }
-
-    List<IThreePidProvider> listUsableProviders(ALookupRequest request) {
-        return listUsableProviders(request, false);
-    }
-
-    List<IThreePidProvider> listUsableProviders(ALookupRequest request, boolean forceRecursive) {
-        List<IThreePidProvider> usableProviders = new ArrayList<>()
-
-        boolean canRecurse = forceRecursive || isAllowedForRecursive(request.getRequester())
-
-        log.info("Host {} allowed for recursion: {}", request.getRequester(), canRecurse)
-        for (IThreePidProvider provider : providers) {
-            if (provider.isEnabled() && (provider.isLocal() || canRecurse || forceRecursive)) {
-                usableProviders.add(provider)
-            }
-        }
-
-        return usableProviders
-    }
-
-    @Override
-    List<IThreePidProvider> getLocalProviders() {
-        return providers.stream().filter(new Predicate<IThreePidProvider>() {
-            @Override
-            boolean test(IThreePidProvider iThreePidProvider) {
-                return iThreePidProvider.isEnabled() && iThreePidProvider.isLocal()
-            }
-        }).collect(Collectors.toList())
-    }
-
-    @Override
-    Optional<SingleLookupReply> find(String medium, String address, boolean recursive) {
-        SingleLookupRequest req = new SingleLookupRequest();
-        req.setType(medium)
-        req.setThreePid(address)
-        req.setRequester("Internal")
-        return find(req, recursive)
-    }
-
-    Optional<SingleLookupReply> find(SingleLookupRequest request, boolean forceRecursive) {
-        for (IThreePidProvider provider : listUsableProviders(request, forceRecursive)) {
-            Optional<SingleLookupReply> lookupDataOpt = provider.find(request)
-            if (lookupDataOpt.isPresent()) {
-                return lookupDataOpt
-            }
-        }
-
-        if (
-        recursiveCfg.getBridge() != null &&
-                recursiveCfg.getBridge().getEnabled() &&
-                (!recursiveCfg.getBridge().getRecursiveOnly() || isAllowedForRecursive(request.getRequester()))
-        ) {
-            log.info("Using bridge failover for lookup")
-            return bridge.find(request)
-        }
-
-        return Optional.empty()
-    }
-
-    @Override
-    Optional<SingleLookupReply> find(SingleLookupRequest request) {
-        return find(request, false)
-    }
-
-    @Override
-    Optional<SingleLookupReply> findRecursive(SingleLookupRequest request) {
-        return find(request, true)
-    }
-
-    @Override
-    List<ThreePidMapping> find(BulkLookupRequest request) {
-        List<ThreePidMapping> mapToDo = new ArrayList<>(request.getMappings())
-        List<ThreePidMapping> mapFoundAll = new ArrayList<>()
-
-        for (IThreePidProvider provider : listUsableProviders(request)) {
-            if (mapToDo.isEmpty()) {
-                log.info("No more mappings to lookup")
-                break
-            } else {
-                log.info("{} mappings remaining overall", mapToDo.size())
-            }
-
-            log.info("Using provider {} for remaining mappings", provider.getClass().getSimpleName())
-            List<ThreePidMapping> mapFound = provider.populate(mapToDo)
-            log.info("Provider {} returned {} mappings", provider.getClass().getSimpleName(), mapFound.size())
-            mapFoundAll.addAll(mapFound)
-            mapToDo.removeAll(mapFound)
-        }
-
-        return mapFoundAll
-    }
-
-}

src/main/groovy/io/kamax/mxisd/mapping/MappingManager.java

@@ -1,175 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.mapping;
-
-import io.kamax.mxisd.exception.BadRequestException;
-import io.kamax.mxisd.lookup.ThreePidValidation;
-import org.apache.commons.lang.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-import org.springframework.stereotype.Component;
-
-import java.time.Instant;
-import java.time.temporal.ChronoUnit;
-import java.util.*;
-
-@Component
-public class MappingManager {
-
-    private Logger log = LoggerFactory.getLogger(MappingManager.class);
-
-    private Map<String, Session> sessions = new HashMap<>();
-    private Timer cleaner;
-
-    MappingManager() {
-        cleaner = new Timer();
-        cleaner.schedule(new TimerTask() {
-            @Override
-            public void run() {
-                List<Session> sList = new ArrayList<>(sessions.values());
-                for (Session s : sList) {
-                    if (s.timestamp.plus(24, ChronoUnit.HOURS).isBefore(Instant.now())) { // TODO config timeout
-                        log.info("Session {} is obsolete, removing", s.sid);
-
-                        sessions.remove(s.sid);
-                    }
-                }
-            }
-        }, 0, 10 * 1000);  // TODO config delay
-    }
-
-    public String create(MappingSession data) {
-        String sid;
-        do {
-            sid = Long.toString(System.currentTimeMillis());
-        } while (sessions.containsKey(sid));
-
-        String threePidHash = data.getMedium() + data.getValue();
-        // TODO think how to handle different requests for the same e-mail
-        Session session = new Session(sid, threePidHash, data);
-        sessions.put(sid, session);
-
-        log.info("Created new session {} to validate {} {}", sid, session.medium, session.address);
-        return sid;
-    }
-
-    public void validate(String sid, String secret, String token) {
-        Session s = sessions.get(sid);
-        if (s == null || !StringUtils.equals(s.secret, secret)) {
-            throw new BadRequestException("sid or secret are not valid");
-        }
-
-        // TODO actually check token
-
-        s.isValidated = true;
-        s.validationTimestamp = Instant.now();
-    }
-
-    public Optional<ThreePidValidation> getValidated(String sid, String secret) {
-        Session s = sessions.get(sid);
-        if (s != null && StringUtils.equals(s.secret, secret)) {
-            return Optional.of(new ThreePidValidation(s.medium, s.address, s.validationTimestamp));
-        }
-
-        return Optional.empty();
-    }
-
-    public void bind(String sid, String secret, String mxid) {
-        Session s = sessions.get(sid);
-        if (s == null || !StringUtils.equals(s.secret, secret)) {
-            throw new BadRequestException("sid or secret are not valid");
-        }
-
-        log.info("Performed bind for mxid {}", mxid);
-        // TODO perform bind, whatever it is
-    }
-
-    private class Session {
-
-        private String sid;
-        private String hash;
-        private Instant timestamp;
-        private Instant validationTimestamp;
-        private boolean isValidated;
-        private String secret;
-        private String medium;
-        private String address;
-
-        public Session(String sid, String hash, MappingSession data) {
-            this.sid = sid;
-            this.hash = hash;
-            timestamp = Instant.now();
-            validationTimestamp = Instant.now();
-            secret = data.getSecret();
-            medium = data.getMedium();
-            address = data.getValue();
-        }
-
-        public Instant getTimestamp() {
-            return timestamp;
-        }
-
-        public void setTimestamp(Instant timestamp) {
-            this.timestamp = timestamp;
-        }
-
-        public Instant getValidationTimestamp() {
-            return validationTimestamp;
-        }
-
-        public void setValidationTimestamp(Instant validationTimestamp) {
-            this.validationTimestamp = validationTimestamp;
-        }
-
-        public boolean isValidated() {
-            return isValidated;
-        }
-
-        public void setValidated(boolean validated) {
-            isValidated = validated;
-        }
-
-        public String getSecret() {
-            return secret;
-        }
-
-        public void setSecret(String secret) {
-            this.secret = secret;
-        }
-
-        public String getMedium() {
-            return medium;
-        }
-
-        public void setMedium(String medium) {
-            this.medium = medium;
-        }
-
-        public String getAddress() {
-            return address;
-        }
-
-        public void setAddress(String address) {
-            this.address = address;
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/signature/SignatureManager.groovy

@@ -1,78 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.signature
-
-import com.google.gson.JsonObject
-import io.kamax.mxisd.config.ServerConfig
-import io.kamax.mxisd.key.KeyManager
-import net.i2p.crypto.eddsa.EdDSAEngine
-import org.json.JSONObject
-import org.springframework.beans.factory.InitializingBean
-import org.springframework.beans.factory.annotation.Autowired
-import org.springframework.stereotype.Component
-
-import java.security.MessageDigest
-
-@Component
-class SignatureManager implements InitializingBean {
-
-    @Autowired
-    private KeyManager keyMgr
-
-    @Autowired
-    private ServerConfig srvCfg
-
-    private EdDSAEngine signEngine
-
-    private String sign(String message) {
-        byte[] signRaw = signEngine.signOneShot(message.getBytes())
-        return Base64.getEncoder().encodeToString(signRaw)
-    }
-
-    JSONObject signMessageJson(String message) {
-        String sign = sign(message)
-
-        JSONObject keySignature = new JSONObject()
-        keySignature.put("ed25519:${keyMgr.getCurrentIndex()}", sign)
-        JSONObject signature = new JSONObject()
-        signature.put("${srvCfg.getName()}", keySignature)
-
-        return signature
-    }
-
-    JsonObject signMessageGson(String message) {
-        String sign = sign(message)
-
-        JsonObject keySignature = new JsonObject()
-        keySignature.addProperty("ed25519:${keyMgr.getCurrentIndex()}", sign)
-        JsonObject signature = new JsonObject()
-        signature.add("${srvCfg.getName()}", keySignature);
-
-        return signature
-    }
-
-    @Override
-    void afterPropertiesSet() throws Exception {
-        signEngine = new EdDSAEngine(MessageDigest.getInstance(keyMgr.getSpecs().getHashAlgorithm()))
-        signEngine.initSign(keyMgr.getPrivateKey(keyMgr.getCurrentIndex()))
-    }
-
-}

src/main/groovy/io/kamax/mxisd/storage/ormlite/OrmLiteSqliteStorage.java

@@ -1,93 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.storage.ormlite;
-
-import com.j256.ormlite.dao.CloseableWrappedIterable;
-import com.j256.ormlite.dao.Dao;
-import com.j256.ormlite.dao.DaoManager;
-import com.j256.ormlite.jdbc.JdbcConnectionSource;
-import com.j256.ormlite.support.ConnectionSource;
-import com.j256.ormlite.table.TableUtils;
-import io.kamax.mxisd.invitation.IThreePidInviteReply;
-import io.kamax.mxisd.storage.IStorage;
-
-import java.io.File;
-import java.io.IOException;
-import java.sql.SQLException;
-import java.util.ArrayList;
-import java.util.Collection;
-import java.util.List;
-
-public class OrmLiteSqliteStorage implements IStorage {
-
-    private Dao<ThreePidInviteIO, String> invDao;
-
-    OrmLiteSqliteStorage(String path) {
-        try {
-            File parent = new File(path).getParentFile();
-            if (!parent.mkdirs() && !parent.isDirectory()) {
-                throw new RuntimeException("Unable to create DB parent directory: " + parent);
-            }
-
-            ConnectionSource connPool = new JdbcConnectionSource("jdbc:sqlite:" + path);
-            invDao = DaoManager.createDao(connPool, ThreePidInviteIO.class);
-            TableUtils.createTableIfNotExists(connPool, ThreePidInviteIO.class);
-        } catch (SQLException e) {
-            throw new RuntimeException(e); // FIXME do better
-        }
-    }
-
-    @Override
-    public Collection<ThreePidInviteIO> getInvites() {
-        try (CloseableWrappedIterable<ThreePidInviteIO> t = invDao.getWrappedIterable()) {
-            List<ThreePidInviteIO> ioList = new ArrayList<>();
-            t.forEach(ioList::add);
-            return ioList;
-        } catch (IOException e) {
-            throw new RuntimeException(e); // FIXME do better
-        }
-    }
-
-    @Override
-    public void insertInvite(IThreePidInviteReply data) {
-        try {
-            int updated = invDao.create(new ThreePidInviteIO(data));
-            if (updated != 1) {
-                throw new RuntimeException("Unexpected row count after DB action: " + updated);
-            }
-        } catch (SQLException e) {
-            throw new RuntimeException(e); // FIXME do better
-        }
-    }
-
-    @Override
-    public void deleteInvite(String id) {
-        try {
-            int updated = invDao.deleteById(id);
-            if (updated != 1) {
-                throw new RuntimeException("Unexpected row count after DB action: " + updated);
-            }
-        } catch (SQLException e) {
-            throw new RuntimeException(e); // FIXME do better
-        }
-    }
-
-}

src/main/groovy/io/kamax/mxisd/storage/ormlite/OrmLiteSqliteStorageBeanFactory.java

@@ -1,76 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.storage.ormlite;
-
-import io.kamax.mxisd.config.SQLiteStorageConfig;
-import io.kamax.mxisd.config.StorageConfig;
-import io.kamax.mxisd.exception.ConfigurationException;
-import io.kamax.mxisd.storage.IStorage;
-import org.apache.commons.lang.StringUtils;
-import org.springframework.beans.factory.FactoryBean;
-import org.springframework.beans.factory.FactoryBeanNotInitializedException;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;
-
-import javax.annotation.PostConstruct;
-
-@Component
-public class OrmLiteSqliteStorageBeanFactory implements FactoryBean<IStorage> {
-
-    @Autowired
-    private StorageConfig storagecfg;
-
-    @Autowired
-    private SQLiteStorageConfig cfg;
-
-    private OrmLiteSqliteStorage storage;
-
-    @PostConstruct
-    private void postConstruct() {
-        if (StringUtils.equals("sqlite", storagecfg.getBackend())) {
-            if (StringUtils.isBlank(cfg.getDatabase())) {
-                throw new ConfigurationException("storage.provider.sqlite.database");
-            }
-
-            storage = new OrmLiteSqliteStorage(cfg.getDatabase());
-        }
-    }
-
-    @Override
-    public IStorage getObject() throws Exception {
-        if (storage == null) {
-            throw new FactoryBeanNotInitializedException();
-        }
-
-        return storage;
-    }
-
-    @Override
-    public Class<?> getObjectType() {
-        return OrmLiteSqliteStorage.class;
-    }
-
-    @Override
-    public boolean isSingleton() {
-        return true;
-    }
-
-}

src/main/java/edazdarevic/commons/net/CIDRUtils.java

@@ -37,6 +37,7 @@ import java.util.List;
  * both IPv4 and IPv6.
  */
 public class CIDRUtils {
+
     private final String cidr;
 
     private InetAddress inetAddress;
@@ -44,7 +45,6 @@ public class CIDRUtils {
     private InetAddress endAddress;
     private final int prefixLength;
 
-
     public CIDRUtils(String cidr) throws UnknownHostException {
 
         this.cidr = cidr;
@@ -66,7 +66,6 @@ public class CIDRUtils {
 
 
     private void calculate() throws UnknownHostException {
-
         ByteBuffer maskBuffer;
         int targetSize;
         if (inetAddress.getAddress().length == 4) {
@@ -120,14 +119,9 @@ public class CIDRUtils {
     }
 
     public String getNetworkAddress() {
-
         return this.startAddress.getHostAddress();
     }
 
-    public String getBroadcastAddress() {
-        return this.endAddress.getHostAddress();
-    }
-
     public boolean isInRange(String ipAddress) throws UnknownHostException {
         InetAddress address = InetAddress.getByName(ipAddress);
         BigInteger start = new BigInteger(1, this.startAddress.getAddress());
@@ -139,4 +133,5 @@ public class CIDRUtils {
 
         return (st == -1 || st == 0) && (te == -1 || te == 0);
     }
+
 }

src/main/java/io/kamax/mxisd/HttpMxisd.java

@@ -0,0 +1,113 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd;
+
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.http.undertow.handler.SaneHandler;
+import io.kamax.mxisd.http.undertow.handler.as.v1.AsNotFoundHandler;
+import io.kamax.mxisd.http.undertow.handler.as.v1.AsTransactionHandler;
+import io.kamax.mxisd.http.undertow.handler.auth.RestAuthHandler;
+import io.kamax.mxisd.http.undertow.handler.auth.v1.LoginGetHandler;
+import io.kamax.mxisd.http.undertow.handler.auth.v1.LoginHandler;
+import io.kamax.mxisd.http.undertow.handler.auth.v1.LoginPostHandler;
+import io.kamax.mxisd.http.undertow.handler.directory.v1.UserDirectorySearchHandler;
+import io.kamax.mxisd.http.undertow.handler.identity.v1.*;
+import io.kamax.mxisd.http.undertow.handler.profile.v1.InternalProfileHandler;
+import io.kamax.mxisd.http.undertow.handler.profile.v1.ProfileHandler;
+import io.kamax.mxisd.http.undertow.handler.status.StatusHandler;
+import io.undertow.Handlers;
+import io.undertow.Undertow;
+import io.undertow.server.HttpHandler;
+
+public class HttpMxisd {
+
+    // Core
+    private Mxisd m;
+
+    // I/O
+    private Undertow httpSrv;
+
+    public HttpMxisd(MxisdConfig cfg) {
+        m = new Mxisd(cfg);
+    }
+
+    public void start() {
+        m.start();
+
+        HttpHandler asNotFoundHandler = SaneHandler.around(new AsNotFoundHandler(m.getAs()));
+        HttpHandler asTxnHandler = SaneHandler.around(new AsTransactionHandler(m.getAs()));
+        HttpHandler storeInvHandler = SaneHandler.around(new StoreInviteHandler(m.getConfig().getServer(), m.getInvitationManager(), m.getKeyManager()));
+        HttpHandler sessValidateHandler = SaneHandler.around(new SessionValidateHandler(m.getSession(), m.getConfig().getServer(), m.getConfig().getView()));
+
+        httpSrv = Undertow.builder().addHttpListener(m.getConfig().getServer().getPort(), "0.0.0.0").setHandler(Handlers.routing()
+
+                // Status endpoints
+                .get(StatusHandler.Path, SaneHandler.around(new StatusHandler()))
+
+                // Authentication endpoints
+                .get(LoginHandler.Path, SaneHandler.around(new LoginGetHandler(m.getAuth(), m.getHttpClient())))
+                .post(LoginHandler.Path, SaneHandler.around(new LoginPostHandler(m.getAuth())))
+                .post(RestAuthHandler.Path, SaneHandler.around(new RestAuthHandler(m.getAuth())))
+
+                // Directory endpoints
+                .post(UserDirectorySearchHandler.Path, SaneHandler.around(new UserDirectorySearchHandler(m.getDirectory())))
+
+                // Key endpoints
+                .get(KeyGetHandler.Path, SaneHandler.around(new KeyGetHandler(m.getKeyManager())))
+                .get(RegularKeyIsValidHandler.Path, SaneHandler.around(new RegularKeyIsValidHandler(m.getKeyManager())))
+                .get(EphemeralKeyIsValidHandler.Path, SaneHandler.around(new EphemeralKeyIsValidHandler()))
+
+                // Identity endpoints
+                .get(HelloHandler.Path, SaneHandler.around(new HelloHandler()))
+                .get(SingleLookupHandler.Path, SaneHandler.around(new SingleLookupHandler(m.getIdentity(), m.getSign())))
+                .post(BulkLookupHandler.Path, SaneHandler.around(new BulkLookupHandler(m.getIdentity())))
+                .post(StoreInviteHandler.Path, storeInvHandler)
+                .post(SessionStartHandler.Path, SaneHandler.around(new SessionStartHandler(m.getSession())))
+                .get(SessionValidateHandler.Path, sessValidateHandler)
+                .post(SessionValidateHandler.Path, sessValidateHandler)
+                .get(SessionTpidGetValidatedHandler.Path, SaneHandler.around(new SessionTpidGetValidatedHandler(m.getSession())))
+                .post(SessionTpidBindHandler.Path, SaneHandler.around(new SessionTpidBindHandler(m.getSession(), m.getInvitationManager())))
+                .post(SessionTpidUnbindHandler.Path, SaneHandler.around(new SessionTpidUnbindHandler(m.getSession())))
+
+                // Profile endpoints
+                .get(ProfileHandler.Path, SaneHandler.around(new ProfileHandler(m.getProfile())))
+                .get(InternalProfileHandler.Path, SaneHandler.around(new InternalProfileHandler(m.getProfile())))
+
+                // Application Service endpoints
+                .get("/_matrix/app/v1/users/**", asNotFoundHandler)
+                .get("/users/**", asNotFoundHandler) // Legacy endpoint
+                .get("/_matrix/app/v1/rooms/**", asNotFoundHandler)
+                .get("/rooms/**", asNotFoundHandler) // Legacy endpoint
+                .put(AsTransactionHandler.Path, asTxnHandler)
+                .put("/transactions/{" + AsTransactionHandler.ID + "}", asTxnHandler) // Legacy endpoint
+
+        ).build();
+
+        httpSrv.start();
+    }
+
+    public void stop() {
+        httpSrv.stop();
+
+        m.stop();
+    }
+
+}

src/main/java/io/kamax/mxisd/Mxisd.java

@@ -0,0 +1,172 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd;
+
+import io.kamax.matrix.crypto.KeyManager;
+import io.kamax.matrix.crypto.SignatureManager;
+import io.kamax.mxisd.as.AppSvcManager;
+import io.kamax.mxisd.auth.AuthManager;
+import io.kamax.mxisd.auth.AuthProviders;
+import io.kamax.mxisd.backend.IdentityStoreSupplier;
+import io.kamax.mxisd.backend.sql.synapse.Synapse;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.crypto.CryptoFactory;
+import io.kamax.mxisd.directory.DirectoryManager;
+import io.kamax.mxisd.directory.DirectoryProviders;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.dns.FederationDnsOverwrite;
+import io.kamax.mxisd.invitation.InvitationManager;
+import io.kamax.mxisd.lookup.ThreePidProviders;
+import io.kamax.mxisd.lookup.fetcher.IRemoteIdentityServerFetcher;
+import io.kamax.mxisd.lookup.provider.BridgeFetcher;
+import io.kamax.mxisd.lookup.provider.RemoteIdentityServerFetcher;
+import io.kamax.mxisd.lookup.strategy.LookupStrategy;
+import io.kamax.mxisd.lookup.strategy.RecursivePriorityLookupStrategy;
+import io.kamax.mxisd.notification.NotificationHandlerSupplier;
+import io.kamax.mxisd.notification.NotificationHandlers;
+import io.kamax.mxisd.notification.NotificationManager;
+import io.kamax.mxisd.profile.ProfileManager;
+import io.kamax.mxisd.profile.ProfileProviders;
+import io.kamax.mxisd.session.SessionManager;
+import io.kamax.mxisd.storage.IStorage;
+import io.kamax.mxisd.storage.ormlite.OrmLiteSqlStorage;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+
+import java.util.ServiceLoader;
+
+public class Mxisd {
+
+    protected MxisdConfig cfg;
+
+    protected CloseableHttpClient httpClient;
+    protected IRemoteIdentityServerFetcher srvFetcher;
+
+    protected IStorage store;
+
+    protected KeyManager keyMgr;
+    protected SignatureManager signMgr;
+
+    // Features
+    protected AuthManager authMgr;
+    protected DirectoryManager dirMgr;
+    protected LookupStrategy idStrategy;
+    protected InvitationManager invMgr;
+    protected ProfileManager pMgr;
+    protected AppSvcManager asHander;
+    protected SessionManager sessMgr;
+    protected NotificationManager notifMgr;
+
+    public Mxisd(MxisdConfig cfg) {
+        this.cfg = cfg.build();
+    }
+
+    protected void build() {
+        httpClient = HttpClients.custom()
+                .setUserAgent("mxisd")
+                .setMaxConnPerRoute(Integer.MAX_VALUE)
+                .setMaxConnTotal(Integer.MAX_VALUE)
+                .build();
+
+        srvFetcher = new RemoteIdentityServerFetcher(httpClient);
+
+        store = new OrmLiteSqlStorage(cfg);
+        keyMgr = CryptoFactory.getKeyManager(cfg.getKey());
+        signMgr = CryptoFactory.getSignatureManager(keyMgr, cfg.getServer());
+        ClientDnsOverwrite clientDns = new ClientDnsOverwrite(cfg.getDns().getOverwrite());
+        FederationDnsOverwrite fedDns = new FederationDnsOverwrite(cfg.getDns().getOverwrite());
+        Synapse synapse = new Synapse(cfg.getSynapseSql());
+        BridgeFetcher bridgeFetcher = new BridgeFetcher(cfg.getLookup().getRecursive().getBridge(), srvFetcher);
+
+        ServiceLoader.load(IdentityStoreSupplier.class).iterator().forEachRemaining(p -> p.accept(this));
+        ServiceLoader.load(NotificationHandlerSupplier.class).iterator().forEachRemaining(p -> p.accept(this));
+
+        idStrategy = new RecursivePriorityLookupStrategy(cfg.getLookup(), ThreePidProviders.get(), bridgeFetcher);
+        pMgr = new ProfileManager(ProfileProviders.get(), clientDns, httpClient);
+        notifMgr = new NotificationManager(cfg.getNotification(), NotificationHandlers.get());
+        sessMgr = new SessionManager(cfg.getSession(), cfg.getMatrix(), store, notifMgr, idStrategy, httpClient);
+        invMgr = new InvitationManager(cfg.getInvite(), store, idStrategy, signMgr, fedDns, notifMgr);
+        authMgr = new AuthManager(cfg, AuthProviders.get(), idStrategy, invMgr, clientDns, httpClient);
+        dirMgr = new DirectoryManager(cfg.getDirectory(), clientDns, httpClient, DirectoryProviders.get());
+        asHander = new AppSvcManager(cfg, store, pMgr, notifMgr, synapse);
+    }
+
+    public MxisdConfig getConfig() {
+        return cfg;
+    }
+
+    public CloseableHttpClient getHttpClient() {
+        return httpClient;
+    }
+
+    public IRemoteIdentityServerFetcher getServerFetcher() {
+        return srvFetcher;
+    }
+
+    public KeyManager getKeyManager() {
+        return keyMgr;
+    }
+
+    public InvitationManager getInvitationManager() {
+        return invMgr;
+    }
+
+    public LookupStrategy getIdentity() {
+        return idStrategy;
+    }
+
+    public AuthManager getAuth() {
+        return authMgr;
+    }
+
+    public SessionManager getSession() {
+        return sessMgr;
+    }
+
+    public DirectoryManager getDirectory() {
+        return dirMgr;
+    }
+
+    public ProfileManager getProfile() {
+        return pMgr;
+    }
+
+    public SignatureManager getSign() {
+        return signMgr;
+    }
+
+    public AppSvcManager getAs() {
+        return asHander;
+    }
+
+    public NotificationManager getNotif() {
+        return notifMgr;
+    }
+
+    public void start() {
+        build();
+    }
+
+    public void stop() {
+        // no-op
+    }
+
+}

src/main/java/io/kamax/mxisd/MxisdStandaloneExec.java

@@ -0,0 +1,69 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd;
+
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.config.YamlConfigLoader;
+import org.apache.commons.lang3.StringUtils;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Iterator;
+import java.util.Objects;
+
+public class MxisdStandaloneExec {
+
+    public static void main(String[] args) throws IOException {
+        MxisdConfig cfg = null;
+
+        Iterator<String> argsIt = Arrays.asList(args).iterator();
+        while (argsIt.hasNext()) {
+            String arg = argsIt.next();
+            if (StringUtils.equals("-c", arg)) {
+                String cfgFile = argsIt.next();
+                cfg = YamlConfigLoader.loadFromFile(cfgFile);
+                System.out.println("Loaded configuration from " + cfgFile);
+            } else {
+                System.out.println("Invalid argument: " + arg);
+                System.exit(1);
+            }
+        }
+
+        if (Objects.isNull(cfg)) {
+            cfg = YamlConfigLoader.tryLoadFromFile("mxisd.yaml").orElseGet(MxisdConfig::new);
+        }
+
+        try {
+            HttpMxisd mxisd = new HttpMxisd(cfg);
+            Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+                mxisd.stop();
+                System.out.println("------------- mxisd stopped -------------");
+            }));
+            mxisd.start();
+
+            System.out.println("------------- mxisd started -------------");
+        } catch (Throwable t) {
+            t.printStackTrace();
+            System.exit(1);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/UserID.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2017 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -30,6 +30,10 @@ public class UserID {
         // stub for (de)serialization
     }
 
+    public UserID(UserIdType type, String value) {
+        this(type.getId(), value);
+    }
+
     public UserID(String type, String value) {
         this.type = type;
         this.value = value;

src/main/java/io/kamax/mxisd/UserIdType.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2017 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as

src/main/java/io/kamax/mxisd/as/AppSvcManager.java

@@ -0,0 +1,223 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePidMedium;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._ThreePid;
+import io.kamax.matrix.event.EventKey;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.backend.sql.synapse.Synapse;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.exception.HttpMatrixException;
+import io.kamax.mxisd.exception.NotAllowedException;
+import io.kamax.mxisd.notification.NotificationManager;
+import io.kamax.mxisd.profile.ProfileManager;
+import io.kamax.mxisd.storage.IStorage;
+import io.kamax.mxisd.storage.ormlite.dao.ASTransactionDao;
+import io.kamax.mxisd.util.GsonParser;
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.InputStream;
+import java.time.Instant;
+import java.util.*;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.stream.Collectors;
+
+public class AppSvcManager {
+
+    private transient final Logger log = LoggerFactory.getLogger(AppSvcManager.class);
+
+    private final GsonParser parser;
+
+    private MatrixConfig cfg;
+    private IStorage store;
+    private ProfileManager profiler;
+    private NotificationManager notif;
+    private Synapse synapse;
+
+    private Map<String, CompletableFuture<String>> transactionsInProgress;
+
+    public AppSvcManager(MxisdConfig cfg, IStorage store, ProfileManager profiler, NotificationManager notif, Synapse synapse) {
+        this.cfg = cfg.getMatrix();
+        this.store = store;
+        this.profiler = profiler;
+        this.notif = notif;
+        this.synapse = synapse;
+
+        parser = new GsonParser();
+        transactionsInProgress = new ConcurrentHashMap<>();
+    }
+
+    public AppSvcManager withToken(String token) {
+        if (StringUtils.isBlank(token)) {
+            throw new HttpMatrixException(401, "M_UNAUTHORIZED", "No HS token");
+        }
+
+        if (!StringUtils.equals(cfg.getListener().getToken().getHs(), token)) {
+            throw new NotAllowedException("Invalid HS token");
+        }
+
+        return this;
+    }
+
+    public CompletableFuture<String> processTransaction(String txnId, InputStream is) {
+        if (StringUtils.isEmpty(txnId)) {
+            throw new IllegalArgumentException("Transaction ID cannot be empty");
+        }
+
+        synchronized (this) {
+            Optional<ASTransactionDao> dao = store.getTransactionResult(cfg.getListener().getLocalpart(), txnId);
+            if (dao.isPresent()) {
+                log.info("AS Transaction {} already processed - returning computed result", txnId);
+                return CompletableFuture.completedFuture(dao.get().getResult());
+            }
+
+            CompletableFuture<String> f = transactionsInProgress.get(txnId);
+            if (Objects.nonNull(f)) {
+                log.info("Returning future for transaction {}", txnId);
+                return f;
+            }
+
+            transactionsInProgress.put(txnId, new CompletableFuture<>());
+        }
+
+        CompletableFuture<String> future = transactionsInProgress.get(txnId);
+
+        Instant start = Instant.now();
+        log.info("Processing AS Transaction {}: start", txnId);
+        try {
+            List<JsonObject> events = GsonUtil.asList(GsonUtil.getArray(parser.parse(is), "events"), JsonObject.class);
+            is.close();
+            log.debug("{} event(s) parsed", events.size());
+
+            processTransaction(events);
+
+            Instant end = Instant.now();
+            String result = "{}";
+
+            try {
+                log.info("Saving transaction details to store");
+                store.insertTransactionResult(cfg.getListener().getLocalpart(), txnId, end, result);
+            } finally {
+                log.debug("Removing CompletedFuture from transaction map");
+                transactionsInProgress.remove(txnId);
+            }
+
+            log.info("Processed AS transaction {} in {} ms", txnId, (Instant.now().toEpochMilli() - start.toEpochMilli()));
+            future.complete(result);
+        } catch (Exception e) {
+            log.error("Unable to properly process transaction {}", txnId, e);
+            future.completeExceptionally(e);
+        }
+
+        log.info("Processing AS Transaction {}: end", txnId);
+        return future;
+    }
+
+    public void processTransaction(List<JsonObject> eventsJson) {
+        log.info("Processing transaction events: start");
+
+        eventsJson.forEach(ev -> {
+            String evId = EventKey.Id.getStringOrNull(ev);
+            if (StringUtils.isBlank(evId)) {
+                log.warn("Event has no ID, skipping");
+                log.debug("Event:\n{}", GsonUtil.getPrettyForLog(ev));
+                return;
+            }
+            log.debug("Event {}: processing start", evId);
+
+            String roomId = EventKey.RoomId.getStringOrNull(ev);
+            if (StringUtils.isBlank(roomId)) {
+                log.debug("Event has no room ID, skipping");
+                return;
+            }
+
+            String senderId = EventKey.Sender.getStringOrNull(ev);
+            if (StringUtils.isBlank(senderId)) {
+                log.debug("Event has no sender ID, skipping");
+                return;
+            }
+            _MatrixID sender = MatrixID.asAcceptable(senderId);
+            log.debug("Sender: {}", senderId);
+
+            if (!StringUtils.equals("m.room.member", GsonUtil.getStringOrNull(ev, "type"))) {
+                log.debug("This is not a room membership event, skipping");
+                return;
+            }
+
+            if (!StringUtils.equals("invite", GsonUtil.getStringOrNull(ev, "membership"))) {
+                log.debug("This is not an invite event, skipping");
+                return;
+            }
+
+            String inviteeId = EventKey.StateKey.getStringOrNull(ev);
+            if (StringUtils.isBlank(inviteeId)) {
+                log.warn("Invalid event: No invitee ID, skipping");
+                return;
+            }
+
+            _MatrixID invitee = MatrixID.asAcceptable(inviteeId);
+            if (!StringUtils.equals(invitee.getDomain(), cfg.getDomain())) {
+                log.debug("Ignoring invite for {}: not a local user");
+                return;
+            }
+
+            log.info("Got invite from {} to {}", senderId, inviteeId);
+
+            boolean wasSent = false;
+            List<_ThreePid> tpids = profiler.getThreepids(invitee).stream()
+                    .filter(tpid -> ThreePidMedium.Email.is(tpid.getMedium()))
+                    .collect(Collectors.toList());
+            log.info("Found {} email(s) in identity store for {}", tpids.size(), inviteeId);
+
+            for (_ThreePid tpid : tpids) {
+                log.info("Found Email to notify about room invitation: {}", tpid.getAddress());
+                Map<String, String> properties = new HashMap<>();
+                profiler.getDisplayName(sender).ifPresent(name -> properties.put("sender_display_name", name));
+                try {
+                    synapse.getRoomName(roomId).ifPresent(name -> properties.put("room_name", name));
+                } catch (RuntimeException e) {
+                    log.warn("Could not fetch room name", e);
+                    log.info("Unable to fetch room name: Did you integrate your Homeserver as documented?");
+                }
+
+                IMatrixIdInvite inv = new MatrixIdInvite(roomId, sender, invitee, tpid.getMedium(), tpid.getAddress(), properties);
+                notif.sendForInvite(inv);
+                log.info("Notification for invite of {} sent to {}", inviteeId, tpid.getAddress());
+                wasSent = true;
+            }
+
+            log.info("Was notification sent? {}", wasSent);
+
+            log.debug("Event {}: processing end", evId);
+        });
+
+        log.info("Processing transaction events: end");
+    }
+
+}

src/main/java/io/kamax/mxisd/as/IMatrixIdInvite.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,14 +18,13 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.invitation.sender;
+package io.kamax.mxisd.as;
 
-import io.kamax.mxisd.invitation.IThreePidInviteReply;
+import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.invitation.IThreePidInvite;
 
-public interface IInviteSender {
+public interface IMatrixIdInvite extends IThreePidInvite {
 
-    String getMedium();
+    _MatrixID getInvitee();
-
-    void send(IThreePidInviteReply invite);
 
 }

src/main/java/io/kamax/mxisd/as/MatrixIdInvite.java

@@ -0,0 +1,77 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.as;
+
+import io.kamax.matrix._MatrixID;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+public class MatrixIdInvite implements IMatrixIdInvite {
+
+    private String roomId;
+    private _MatrixID sender;
+    private _MatrixID invitee;
+    private String medium;
+    private String address;
+    private Map<String, String> properties;
+
+    public MatrixIdInvite(String roomId, _MatrixID sender, _MatrixID invitee, String medium, String address, Map<String, String> properties) {
+        this.roomId = Objects.requireNonNull(roomId);
+        this.sender = Objects.requireNonNull(sender);
+        this.invitee = Objects.requireNonNull(invitee);
+        this.medium = Objects.requireNonNull(medium);
+        this.address = Objects.requireNonNull(address);
+        this.properties = new HashMap<>(Objects.requireNonNull(properties));
+    }
+
+    @Override
+    public _MatrixID getSender() {
+        return sender;
+    }
+
+    @Override
+    public String getMedium() {
+        return medium;
+    }
+
+    @Override
+    public String getAddress() {
+        return address;
+    }
+
+    @Override
+    public _MatrixID getInvitee() {
+        return invitee;
+    }
+
+    @Override
+    public String getRoomId() {
+        return roomId;
+    }
+
+    @Override
+    public Map<String, String> getProperties() {
+        return properties;
+    }
+
+}

src/main/java/io/kamax/mxisd/auth/AuthManager.java

@@ -0,0 +1,274 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.auth;
+
+import com.google.gson.Gson;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import com.google.gson.JsonSyntaxException;
+import com.google.i18n.phonenumbers.NumberParseException;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import com.google.i18n.phonenumbers.Phonenumber;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._ThreePid;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import io.kamax.mxisd.config.AuthenticationConfig;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.RemoteLoginException;
+import io.kamax.mxisd.invitation.InvitationManager;
+import io.kamax.mxisd.lookup.ThreePidMapping;
+import io.kamax.mxisd.lookup.strategy.LookupStrategy;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.HttpEntity;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URI;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Objects;
+
+public class AuthManager {
+
+    private static final String TypeKey = "type";
+    private static final String UserKey = "user";
+    private static final String IdentifierKey = "identifier";
+    private static final String ThreepidMediumKey = "medium";
+    private static final String ThreepidAddressKey = "address";
+    private static final String UserIdTypeValue = "m.id.user";
+    private static final String ThreepidTypeValue = "m.id.thirdparty";
+
+    private transient final Logger log = LoggerFactory.getLogger(AuthManager.class);
+    private final Gson gson = GsonUtil.get(); // FIXME replace
+
+    private List<AuthenticatorProvider> providers;
+    private MatrixConfig mxCfg;
+    private AuthenticationConfig cfg;
+    private InvitationManager invMgr;
+    private ClientDnsOverwrite dns;
+    private LookupStrategy strategy;
+    private CloseableHttpClient client;
+
+    public AuthManager(
+            MxisdConfig cfg,
+            List<? extends AuthenticatorProvider> providers,
+            LookupStrategy strategy,
+            InvitationManager invMgr,
+            ClientDnsOverwrite dns,
+            CloseableHttpClient client
+    ) {
+        this.cfg = cfg.getAuth();
+        this.mxCfg = cfg.getMatrix();
+        this.providers = new ArrayList<>(providers);
+        this.strategy = strategy;
+        this.invMgr = invMgr;
+        this.dns = dns;
+        this.client = client;
+    }
+
+    public String resolveProxyUrl(URI target) {
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
+
+    public UserAuthResult authenticate(String id, String password) {
+        _MatrixID mxid = MatrixID.asAcceptable(id);
+        for (AuthenticatorProvider provider : providers) {
+            if (!provider.isEnabled()) {
+                continue;
+            }
+
+            log.info("Attempting authentication with store {}", provider.getClass().getSimpleName());
+
+            BackendAuthResult result = provider.authenticate(mxid, password);
+            if (result.isSuccess()) {
+                String mxId;
+                if (UserIdType.Localpart.is(result.getId().getType())) {
+                    mxId = MatrixID.from(result.getId().getValue(), mxCfg.getDomain()).acceptable().getId();
+                } else if (UserIdType.MatrixID.is(result.getId().getType())) {
+                    mxId = MatrixID.asAcceptable(result.getId().getValue()).getId();
+                } else {
+                    log.warn("Unsupported User ID type {} for backend {}", result.getId().getType(), provider.getClass().getSimpleName());
+                    continue;
+                }
+
+                UserAuthResult authResult = new UserAuthResult().success(result.getProfile().getDisplayName());
+                for (_ThreePid pid : result.getProfile().getThreePids()) {
+                    authResult.withThreePid(pid.getMedium(), pid.getAddress());
+                }
+                log.info("{} was authenticated by {}, publishing 3PID mappings, if any", id, provider.getClass().getSimpleName());
+                for (ThreePid pid : authResult.getThreePids()) {
+                    log.info("Processing {} for {}", pid, id);
+                    invMgr.publishMappingIfInvited(new ThreePidMapping(pid, mxId));
+                }
+
+                invMgr.lookupMappingsForInvites();
+
+                return authResult;
+            }
+        }
+
+        return new UserAuthResult().failure();
+    }
+
+    public String proxyLogin(URI target, String body) {
+        JsonObject reqJsonObject = io.kamax.matrix.json.GsonUtil.parseObj(body);
+
+        GsonUtil.findObj(reqJsonObject, IdentifierKey).ifPresent(obj -> {
+            GsonUtil.findString(obj, TypeKey).ifPresent(type -> {
+                if (StringUtils.equals(type, UserIdTypeValue)) {
+                    log.info("Login request is User ID type");
+
+                    if (cfg.getRewrite().getUser().getRules().isEmpty()) {
+                        log.info("No User ID rewrite rules to apply");
+                    } else {
+                        log.info("User ID rewrite rules: checking for a match");
+
+                        String userId = GsonUtil.getStringOrThrow(obj, UserKey);
+                        for (AuthenticationConfig.Rule m : cfg.getRewrite().getUser().getRules()) {
+                            if (m.getPattern().matcher(userId).matches()) {
+                                log.info("Found matching pattern, resolving to 3PID with medium {}", m.getMedium());
+
+                                // Remove deprecated login info on the top object if exists to avoid duplication
+                                reqJsonObject.remove(UserKey);
+                                obj.addProperty(TypeKey, ThreepidTypeValue);
+                                obj.addProperty(ThreepidMediumKey, m.getMedium());
+                                obj.addProperty(ThreepidAddressKey, userId);
+
+                                log.info("Rewrite to 3PID done");
+                            }
+                        }
+
+                        log.info("User ID rewrite rules: done checking rules");
+                    }
+                }
+            });
+        });
+
+        GsonUtil.findObj(reqJsonObject, IdentifierKey).ifPresent(obj -> {
+            GsonUtil.findString(obj, TypeKey).ifPresent(type -> {
+                if (StringUtils.equals(type, ThreepidTypeValue)) {
+                    // Remove deprecated login info if exists to avoid duplication
+                    reqJsonObject.remove(ThreepidMediumKey);
+                    reqJsonObject.remove(ThreepidAddressKey);
+
+                    GsonUtil.findPrimitive(obj, ThreepidMediumKey).ifPresent(medium -> {
+                        GsonUtil.findPrimitive(obj, ThreepidAddressKey).ifPresent(address -> {
+                            log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
+                            strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
+                                obj.remove(ThreepidMediumKey);
+                                obj.remove(ThreepidAddressKey);
+                                obj.addProperty(TypeKey, UserIdTypeValue);
+                                obj.addProperty(UserKey, lookupDataOpt.getMxid().getLocalPart());
+                            });
+                        });
+                    });
+                }
+
+                if (StringUtils.equals(type, "m.id.phone")) {
+                    // Remove deprecated login info if exists to avoid duplication
+                    reqJsonObject.remove(ThreepidMediumKey);
+                    reqJsonObject.remove(ThreepidAddressKey);
+
+                    GsonUtil.findPrimitive(obj, "number").ifPresent(number -> {
+                        GsonUtil.findPrimitive(obj, "country").ifPresent(country -> {
+                            log.info("Login request with phone '{}'-'{}'", country.getAsString(), number.getAsString());
+                            try {
+                                PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+                                Phonenumber.PhoneNumber phoneNumber = phoneUtil.parse(number.getAsString(), country.getAsString());
+                                String msisdn = phoneUtil.format(phoneNumber, PhoneNumberUtil.PhoneNumberFormat.E164).replace("+", "");
+                                String medium = "msisdn";
+                                strategy.findLocal(medium, msisdn).ifPresent(lookupDataOpt -> {
+                                    obj.remove("country");
+                                    obj.remove("number");
+                                    obj.addProperty(TypeKey, UserIdTypeValue);
+                                    obj.addProperty(UserKey, lookupDataOpt.getMxid().getLocalPart());
+                                });
+                            } catch (NumberParseException e) {
+                                log.error("Not a valid phone number");
+                                throw new RuntimeException(e);
+                            }
+                        });
+                    });
+                }
+            });
+        });
+
+        // invoke 'login' on homeserver
+        HttpPost httpPost = RestClientUtils.post(resolveProxyUrl(target), gson, reqJsonObject);
+        try (CloseableHttpResponse httpResponse = client.execute(httpPost)) {
+            // check http status
+            int status = httpResponse.getStatusLine().getStatusCode();
+            log.info("http status = {}", status);
+            if (status != 200) {
+                // try to get possible json error message from response
+                // otherwise just get returned plain error message
+                String errcode = String.valueOf(httpResponse.getStatusLine().getStatusCode());
+                String error = EntityUtils.toString(httpResponse.getEntity());
+                if (httpResponse.getEntity() != null) {
+                    try {
+                        JsonObject bodyJson = new JsonParser().parse(error).getAsJsonObject();
+                        if (bodyJson.has("errcode")) {
+                            errcode = bodyJson.get("errcode").getAsString();
+                        }
+                        if (bodyJson.has("error")) {
+                            error = bodyJson.get("error").getAsString();
+                        }
+                        throw new RemoteLoginException(status, errcode, error, bodyJson);
+                    } catch (JsonSyntaxException e) {
+                        log.warn("Response body is not JSON");
+                    }
+                }
+                throw new RemoteLoginException(status, errcode, error);
+            }
+
+            // return response
+            HttpEntity entity = httpResponse.getEntity();
+            if (Objects.isNull(entity)) {
+                log.warn("Expected HS to return data but got nothing");
+                return "";
+            } else {
+                return IOUtils.toString(httpResponse.getEntity().getContent(), StandardCharsets.UTF_8);
+            }
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/auth/AuthProviders.java

@@ -0,0 +1,42 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.auth;
+
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.function.Supplier;
+import java.util.stream.Collectors;
+
+public class AuthProviders {
+
+    private static final List<Supplier<? extends AuthenticatorProvider>> suppliers = new ArrayList<>();
+
+    public static void register(Supplier<? extends AuthenticatorProvider> supplier) {
+        suppliers.add(supplier);
+    }
+
+    public static List<? extends AuthenticatorProvider> get() {
+        return suppliers.stream().map(Supplier::get).collect(Collectors.toList());
+    }
+
+}

src/main/java/io/kamax/mxisd/auth/UserAuthResult.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2017 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -20,31 +20,30 @@
 
 package io.kamax.mxisd.auth;
 
-import io.kamax.matrix.ThreePidMedium;
+import io.kamax.matrix.ThreePid;
-import io.kamax.mxisd.ThreePid;
 
-import java.util.ArrayList;
 import java.util.Collections;
-import java.util.List;
+import java.util.HashSet;
+import java.util.Set;
 
 public class UserAuthResult {
 
     private boolean success;
-    private String mxid;
     private String displayName;
-    private List<ThreePid> threePids = new ArrayList<>();
+    private String photo;
+    private Set<ThreePid> threePids = new HashSet<>();
 
     public UserAuthResult failure() {
         success = false;
-        mxid = null;
         displayName = null;
+        photo = null;
+        threePids.clear();
 
         return this;
     }
 
-    public UserAuthResult success(String mxid, String displayName) {
+    public UserAuthResult success(String displayName) {
         setSuccess(true);
-        setMxid(mxid);
         setDisplayName(displayName);
 
         return this;
@@ -58,14 +57,6 @@ public class UserAuthResult {
         this.success = success;
     }
 
-    public String getMxid() {
-        return mxid;
-    }
-
-    public void setMxid(String mxid) {
-        this.mxid = mxid;
-    }
-
     public String getDisplayName() {
         return displayName;
     }
@@ -74,8 +65,12 @@ public class UserAuthResult {
         this.displayName = displayName;
     }
 
-    public UserAuthResult withThreePid(ThreePidMedium medium, String address) {
+    public String getPhoto() {
-        return withThreePid(medium.getId(), address);
+        return photo;
+    }
+
+    public void setPhoto(String photo) {
+        this.photo = photo;
     }
 
     public UserAuthResult withThreePid(String medium, String address) {
@@ -84,8 +79,8 @@ public class UserAuthResult {
         return this;
     }
 
-    public List<ThreePid> getThreePids() {
+    public Set<ThreePid> getThreePids() {
-        return Collections.unmodifiableList(threePids);
+        return Collections.unmodifiableSet(threePids);
     }
 
 }

src/main/java/io/kamax/mxisd/auth/provider/AuthenticatorProvider.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2017 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as

src/main/java/io/kamax/mxisd/auth/provider/BackendAuthResult.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2017 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -20,25 +20,29 @@
 
 package io.kamax.mxisd.auth.provider;
 
-import io.kamax.mxisd.ThreePid;
+import io.kamax.matrix.ThreePid;
 import io.kamax.mxisd.UserID;
 import io.kamax.mxisd.UserIdType;
 
-import java.util.ArrayList;
+import java.util.HashSet;
-import java.util.List;
+import java.util.Set;
 
 public class BackendAuthResult {
 
     public static class BackendAuthProfile {
 
         private String displayName;
-        private List<ThreePid> threePids = new ArrayList<>();
+        private Set<ThreePid> threePids = new HashSet<>();
 
         public String getDisplayName() {
             return displayName;
         }
 
-        public List<ThreePid> getThreePids() {
+        public void setDisplayName(String displayName) {
+            this.displayName = displayName;
+        }
+
+        public Set<ThreePid> getThreePids() {
             return threePids;
         }
     }
@@ -49,24 +53,34 @@ public class BackendAuthResult {
         return r;
     }
 
+    public void fail() {
+        success = false;
+    }
+
     public static BackendAuthResult success(String id, UserIdType type, String displayName) {
         return success(id, type.getId(), displayName);
     }
 
     public static BackendAuthResult success(String id, String type, String displayName) {
         BackendAuthResult r = new BackendAuthResult();
-        r.success = true;
+        r.succeed(id, type, displayName);
-        r.id = new UserID(type, id);
-        r.profile = new BackendAuthProfile();
-        r.profile.displayName = displayName;
-
         return r;
     }
 
+    public void succeed(String id, String type, String displayName) {
+        this.success = true;
+        this.id = new UserID(type, id);
+        this.profile.displayName = displayName;
+    }
+
     private Boolean success;
     private UserID id;
     private BackendAuthProfile profile = new BackendAuthProfile();
 
+    public void setSuccess(boolean success) {
+        this.success = success;
+    }
+
     public Boolean isSuccess() {
         return success;
     }
@@ -75,6 +89,10 @@ public class BackendAuthResult {
         return id;
     }
 
+    public void setId(UserID id) {
+        this.id = id;
+    }
+
     public BackendAuthProfile getProfile() {
         return profile;
     }

src/main/java/io/kamax/mxisd/backend/IdentityStoreSupplier.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,18 +18,12 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.mapping;
+package io.kamax.mxisd.backend;
 
-public interface MappingSession {
+import io.kamax.mxisd.Mxisd;
 
-    String getServer();
+import java.util.function.Consumer;
 
-    String getSecret();
+public interface IdentityStoreSupplier extends Consumer<Mxisd> {
-
-    int getAttempt();
-
-    String getMedium();
-
-    String getValue();
 
 }

src/main/java/io/kamax/mxisd/backend/exec/ExecAuthResult.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sarl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,16 +18,20 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd
+package io.kamax.mxisd.backend.exec;
 
-import org.springframework.boot.SpringApplication
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
-import org.springframework.boot.autoconfigure.SpringBootApplication
 
-@SpringBootApplication
+public class ExecAuthResult extends BackendAuthResult {
-class MatrixIdentityServerApplication {
 
-    static void main(String[] args) throws Exception {
+    private int exitStatus;
-        SpringApplication.run(MatrixIdentityServerApplication.class, args)
+
+    public int getExitStatus() {
+        return exitStatus;
+    }
+
+    public void setExitStatus(int exitStatus) {
+        this.exitStatus = exitStatus;
     }
 
 }

src/main/java/io/kamax/mxisd/backend/exec/ExecAuthStore.java

@@ -0,0 +1,125 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.exec;
+
+import com.google.gson.JsonObject;
+import com.google.gson.JsonPrimitive;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.UserID;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+import io.kamax.mxisd.backend.rest.RestAuthRequestJson;
+import io.kamax.mxisd.config.ExecConfig;
+import io.kamax.mxisd.exception.InternalServerError;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Objects;
+import java.util.Optional;
+
+public class ExecAuthStore extends ExecStore implements AuthenticatorProvider {
+
+    private transient final Logger log = LoggerFactory.getLogger(ExecAuthStore.class);
+
+    private ExecConfig.Auth cfg;
+
+    public ExecAuthStore(ExecConfig cfg) {
+        this.cfg = Objects.requireNonNull(cfg.getAuth());
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return cfg.isEnabled();
+    }
+
+    @Override
+    public ExecAuthResult authenticate(_MatrixID uId, String password) {
+        Objects.requireNonNull(uId);
+        Objects.requireNonNull(password);
+
+        log.info("Performing authentication for {}", uId.getId());
+
+        ExecAuthResult result = new ExecAuthResult();
+        result.setId(new UserID(UserIdType.Localpart, uId.getLocalPart()));
+
+        Processor<ExecAuthResult> p = new Processor<>(cfg);
+
+        p.addTokenMapper(cfg.getToken().getLocalpart(), uId::getLocalPart);
+        p.addTokenMapper(cfg.getToken().getDomain(), uId::getDomain);
+        p.addTokenMapper(cfg.getToken().getMxid(), uId::getId);
+        p.addTokenMapper(cfg.getToken().getPassword(), () -> password);
+
+        p.addJsonInputTemplate(tokens -> {
+            RestAuthRequestJson json = new RestAuthRequestJson();
+            json.setLocalpart(tokens.getLocalpart());
+            json.setDomain(tokens.getDomain());
+            json.setMxid(tokens.getMxid());
+            json.setPassword(tokens.getPassword());
+            return json;
+        });
+        p.addInputTemplate(PlainType, tokens -> tokens.getLocalpart() + System.lineSeparator() +
+                tokens.getDomain() + System.lineSeparator() +
+                tokens.getMxid() + System.lineSeparator() +
+                tokens.getPassword() + System.lineSeparator()
+        );
+
+        p.withExitHandler(pr -> result.setExitStatus(pr.getExitValue()));
+
+        p.withSuccessHandler(pr -> result.setSuccess(true));
+        p.withSuccessDefault(o -> result);
+        p.addSuccessMapper(JsonType, output -> {
+            JsonObject data = GsonUtil.getObj(GsonUtil.parseObj(output), "auth");
+            GsonUtil.findPrimitive(data, "success")
+                    .map(JsonPrimitive::getAsBoolean)
+                    .ifPresent(result::setSuccess);
+            GsonUtil.findObj(data, "profile")
+                    .flatMap(profile -> GsonUtil.findString(profile, "display_name"))
+                    .ifPresent(v -> result.getProfile().setDisplayName(v));
+
+            return result;
+        });
+        p.addSuccessMapper(PlainType, output -> {
+            String[] lines = output.split("\\R");
+            if (lines.length > 2) {
+                throw new InternalServerError("Exec auth command returned more than 2 lines (" + lines.length + ")");
+            }
+
+            result.setSuccess(Optional.ofNullable(StringUtils.isEmpty(lines[0]) ? null : lines[0])
+                    .map(v -> StringUtils.equalsAnyIgnoreCase(v, "true", "1"))
+                    .orElse(result.isSuccess()));
+
+            if (lines.length == 2) {
+                Optional.ofNullable(StringUtils.isEmpty(lines[1]) ? null : lines[1])
+                        .ifPresent(v -> result.getProfile().setDisplayName(v));
+            }
+
+            return result;
+        });
+
+        p.withFailureHandler(pr -> result.setSuccess(false));
+        p.withFailureDefault(o -> result);
+
+        return p.execute();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/exec/ExecDirectoryStore.java

@@ -0,0 +1,86 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.exec;
+
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.config.ExecConfig;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.directory.DirectoryProvider;
+import io.kamax.mxisd.http.io.UserDirectorySearchRequest;
+import io.kamax.mxisd.http.io.UserDirectorySearchResult;
+import org.apache.commons.lang3.StringUtils;
+
+public class ExecDirectoryStore extends ExecStore implements DirectoryProvider {
+
+    private ExecConfig.Directory cfg;
+    private MatrixConfig mxCfg;
+
+    public ExecDirectoryStore(MxisdConfig cfg) {
+        this(cfg.getExec().getDirectory(), cfg.getMatrix());
+    }
+
+    public ExecDirectoryStore(ExecConfig.Directory cfg, MatrixConfig mxCfg) {
+        this.cfg = cfg;
+        this.mxCfg = mxCfg;
+    }
+
+    private UserDirectorySearchResult search(ExecConfig.Process cfg, UserDirectorySearchRequest request) {
+        if (StringUtils.isEmpty(cfg.getCommand())) {
+            return UserDirectorySearchResult.empty();
+        }
+
+        Processor<UserDirectorySearchResult> p = new Processor<>(cfg);
+
+        p.addJsonInputTemplate(tokens -> new UserDirectorySearchRequest(tokens.getType(), tokens.getQuery()));
+        p.addInputTemplate(PlainType, tokens -> tokens.getType() + System.lineSeparator() + tokens.getQuery());
+
+        p.addTokenMapper(cfg.getToken().getType(), request::getBy);
+        p.addTokenMapper(cfg.getToken().getQuery(), request::getSearchTerm);
+
+        p.addSuccessMapper(JsonType, output -> {
+            if (StringUtils.isBlank(output)) {
+                return UserDirectorySearchResult.empty();
+            }
+
+            UserDirectorySearchResult response = GsonUtil.get().fromJson(output, UserDirectorySearchResult.class);
+            for (UserDirectorySearchResult.Result result : response.getResults()) {
+                result.setUserId(MatrixID.asAcceptable(result.getUserId(), mxCfg.getDomain()).getId());
+            }
+            return response;
+        });
+        p.withFailureDefault(output -> new UserDirectorySearchResult());
+
+        return p.execute();
+    }
+
+    @Override
+    public UserDirectorySearchResult searchByDisplayName(String query) {
+        return search(cfg.getSearch().getByName(), new UserDirectorySearchRequest("name", query));
+    }
+
+    @Override
+    public UserDirectorySearchResult searchBy3pid(String query) {
+        return search(cfg.getSearch().getByName(), new UserDirectorySearchRequest("threepid", query));
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStore.java

@@ -0,0 +1,200 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.exec;
+
+import com.google.gson.JsonArray;
+import com.google.gson.JsonParseException;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.UserID;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.backend.rest.LookupBulkResponseJson;
+import io.kamax.mxisd.backend.rest.LookupSingleResponseJson;
+import io.kamax.mxisd.config.ExecConfig;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.lookup.SingleLookupReply;
+import io.kamax.mxisd.lookup.SingleLookupRequest;
+import io.kamax.mxisd.lookup.ThreePidMapping;
+import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+import java.util.stream.Collectors;
+
+public class ExecIdentityStore extends ExecStore implements IThreePidProvider {
+
+    private transient final Logger log = LoggerFactory.getLogger(ExecIdentityStore.class);
+
+    private final ExecConfig.Identity cfg;
+    private final MatrixConfig mxCfg;
+
+    public ExecIdentityStore(ExecConfig cfg, MatrixConfig mxCfg) {
+        this(cfg.getIdentity(), mxCfg);
+    }
+
+    public ExecIdentityStore(ExecConfig.Identity cfg, MatrixConfig mxCfg) {
+        this.cfg = cfg;
+        this.mxCfg = mxCfg;
+    }
+
+    @Override
+    public boolean isLocal() {
+        return true;
+    }
+
+    @Override
+    public int getPriority() {
+        return cfg.getPriority();
+    }
+
+    private ExecConfig.Process getSingleCfg() {
+        return cfg.getLookup().getSingle();
+    }
+
+    private _MatrixID getUserId(UserID id) {
+        if (Objects.isNull(id)) {
+            throw new JsonParseException("User id key is not present");
+        }
+
+        if (UserIdType.Localpart.is(id.getType())) {
+            return MatrixID.asAcceptable(id.getValue(), mxCfg.getDomain());
+        }
+
+        if (UserIdType.MatrixID.is(id.getType())) {
+            return MatrixID.asAcceptable(id.getValue());
+        }
+
+        throw new InternalServerError("Unknown user type: " + id.getType());
+    }
+
+    @Override
+    public Optional<SingleLookupReply> find(SingleLookupRequest request) {
+        Processor<Optional<SingleLookupReply>> p = new Processor<>();
+        p.withConfig(cfg.getLookup().getSingle());
+
+        p.addTokenMapper(getSingleCfg().getToken().getMedium(), request::getType);
+        p.addTokenMapper(getSingleCfg().getToken().getAddress(), request::getThreePid);
+
+        p.addJsonInputTemplate(tokens -> new ThreePid(tokens.getMedium(), tokens.getAddress()));
+        p.addInputTemplate(PlainType, tokens -> tokens.getMedium()
+                + System.lineSeparator()
+                + tokens.getAddress()
+        );
+
+        p.addSuccessMapper(JsonType, output -> {
+            if (StringUtils.isBlank(output)) {
+                return Optional.empty();
+            }
+
+            return GsonUtil.findObj(GsonUtil.parseObj(output), "lookup")
+                    .filter(obj -> !obj.entrySet().isEmpty())
+                    .map(json -> GsonUtil.get().fromJson(json, LookupSingleResponseJson.class))
+                    .map(lookup -> getUserId(lookup.getId()))
+                    .map(mxId -> new SingleLookupReply(request, mxId));
+        });
+
+        p.addSuccessMapper(PlainType, output -> {
+            String[] lines = output.split("\\R");
+            if (lines.length > 2) {
+                throw new InternalServerError("Exec auth command returned more than 2 lines (" + lines.length + ")");
+            }
+
+            if (lines.length == 1 && StringUtils.isBlank(lines[0])) {
+                return Optional.empty();
+            }
+
+            String type = StringUtils.trimToEmpty(lines.length == 1 ? UserIdType.Localpart.getId() : lines[0]);
+            String value = StringUtils.trimToEmpty(lines.length == 2 ? lines[1] : lines[0]);
+
+            if (UserIdType.Localpart.is(type)) {
+                return Optional.of(new SingleLookupReply(request, MatrixID.asAcceptable(value, mxCfg.getDomain())));
+            }
+
+            if (UserIdType.MatrixID.is(type)) {
+                return Optional.of(new SingleLookupReply(request, MatrixID.asAcceptable(value)));
+            }
+
+            throw new InternalServerError("Invalid user type: " + type);
+        });
+
+        p.withFailureDefault(o -> Optional.empty());
+
+        return p.execute();
+    }
+
+    @Override
+    public List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
+        Processor<List<ThreePidMapping>> p = new Processor<>();
+        p.withConfig(cfg.getLookup().getBulk());
+
+        p.addInput(JsonType, () -> {
+            JsonArray tpids = GsonUtil.asArray(mappings.stream()
+                    .map(mapping -> GsonUtil.get().toJsonTree(new ThreePid(mapping.getMedium(), mapping.getValue())))
+                    .collect(Collectors.toList()));
+            return GsonUtil.get().toJson(GsonUtil.makeObj("lookup", tpids));
+        });
+        p.addInput(PlainType, () -> {
+            StringBuilder input = new StringBuilder();
+            for (ThreePidMapping mapping : mappings) {
+                input.append(mapping.getMedium()).append("\t").append(mapping.getValue()).append(System.lineSeparator());
+            }
+            return input.toString();
+        });
+
+        p.addSuccessMapper(JsonType, output -> {
+            if (StringUtils.isBlank(output)) {
+                return Collections.emptyList();
+            }
+
+            LookupBulkResponseJson response = GsonUtil.get().fromJson(output, LookupBulkResponseJson.class);
+            return response.getLookup().stream().map(item -> {
+                ThreePidMapping mapping = new ThreePidMapping();
+                mapping.setMedium(item.getMedium());
+                mapping.setValue(item.getAddress());
+
+                if (UserIdType.Localpart.is(item.getId().getType())) {
+                    mapping.setValue(MatrixID.asAcceptable(item.getId().getValue(), mxCfg.getDomain()).getId());
+                    return mapping;
+                }
+
+                if (UserIdType.MatrixID.is(item.getId().getType())) {
+                    mapping.setValue(MatrixID.asAcceptable(item.getId().getValue()).getId());
+                    return mapping;
+                }
+
+                throw new InternalServerError("Invalid user type: " + item.getId().getType());
+            }).collect(Collectors.toList());
+        });
+
+        p.withFailureDefault(output -> Collections.emptyList());
+
+        return p.execute();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/exec/ExecIdentityStoreSupplier.java

@@ -0,0 +1,56 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.exec;
+
+import io.kamax.mxisd.Mxisd;
+import io.kamax.mxisd.auth.AuthProviders;
+import io.kamax.mxisd.backend.IdentityStoreSupplier;
+import io.kamax.mxisd.config.MxisdConfig;
+import io.kamax.mxisd.directory.DirectoryProviders;
+import io.kamax.mxisd.lookup.ThreePidProviders;
+import io.kamax.mxisd.profile.ProfileProviders;
+
+public class ExecIdentityStoreSupplier implements IdentityStoreSupplier {
+
+    @Override
+    public void accept(Mxisd mxisd) {
+        accept(mxisd.getConfig());
+    }
+
+    public void accept(MxisdConfig cfg) {
+        if (cfg.getExec().getAuth().isEnabled()) {
+            AuthProviders.register(() -> new ExecAuthStore(cfg.getExec()));
+        }
+
+        if (cfg.getExec().getDirectory().isEnabled()) {
+            DirectoryProviders.register(() -> new ExecDirectoryStore(cfg));
+        }
+
+        if (cfg.getExec().getIdentity().isEnabled()) {
+            ThreePidProviders.register(() -> new ExecIdentityStore(cfg.getExec(), cfg.getMatrix()));
+        }
+
+        if (cfg.getExec().getProfile().isEnabled()) {
+            ProfileProviders.register(() -> new ExecProfileStore(cfg.getExec()));
+        }
+    }
+
+}