Add 3PIDs at registration

A recent change in synapse shown that the various classes handling crypto were broken.
All the crypto code has been refactored in the SDK and the local code has been adapted.

.travis.yml

@@ -1,4 +1,8 @@
-language: groovy
-
-jdk:
-  - oraclejdk8
+language: java
+before_cache:
+  - rm -f  $HOME/.gradle/caches/modules-2/modules-2.lock
+  - rm -fr $HOME/.gradle/caches/*/plugin-resolution/
+cache:
+  directories:
+    - $HOME/.gradle/caches/
+    - $HOME/.gradle/wrapper/

Dockerfile

@@ -1,11 +1,17 @@
 FROM openjdk:8-jre-alpine
 
+RUN apk update && apk add bash && rm -rf /var/lib/apk/* /var/cache/apk/*
+
 VOLUME /etc/mxisd
 VOLUME /var/mxisd
 EXPOSE 8090
 
-ADD build/libs/mxisd.jar /mxisd.jar
-ADD src/docker/start.sh /start.sh
-
 ENV JAVA_OPTS=""
+ENV CONF_FILE_PATH="/etc/mxisd/mxisd.yaml"
+ENV SIGN_KEY_PATH="/var/mxisd/sign.key"
+ENV SQLITE_DATABASE_PATH="/var/mxisd/mxisd.db"
+
 CMD [ "/start.sh" ]
+
+ADD src/docker/start.sh /start.sh
+ADD build/libs/mxisd.jar /mxisd.jar

README.md

@@ -1,259 +1,105 @@
-mxisd - Federated Matrix Identity Server Daemon
------
+mxisd - Federated Matrix Identity Server
+----------------------------------------
 ![Travis-CI build status](https://travis-ci.org/kamax-io/mxisd.svg?branch=master)  
 
 - [Overview](#overview)
 - [Features](#features)
-- [Why use mxisd](#why-use-mxisd)
-- [Quick start](#quick-start)
+- [Use cases](#use-cases)
+- [Getting Started](#getting-started)
 - [Support](#support)
 - [Contribute](#contribute)
 - [FAQ](#faq)
 - [Contact](#contact)
 
 # Overview
-mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with enhanced features.
+mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with [enhanced features](#features).
+As an enhanced Identity service, it implements the [Matrix Identity service API](https://kamax.io/matrix/api/identity_service/unstable.html)
+and several [extra features](#features) that greatly enhance user experience within Matrix.
+It is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built in a
+single coherent product.
   
-It is specifically designed to connect to an Identity store (AD/Samba/LDAP, SQL Database, Web services/application, ...)
-and ease the integration of the Matrix ecosystem with an existing infrastructure, or to build a new one using lasting
-tools.
+mxisd is specifically designed to connect to an existing on-premise Identity store (AD/Samba/LDAP, SQL Database,
+Web services/app, etc.) and ease the integration of a Matrix infrastructure within an existing one.
 
-The core principle of mxisd is to map between Matrix IDs and 3PIDs (Thrid-party Identifiers) for the Homeserver and its
-users. 3PIDs can be anything that identify a user, like:
-- Full name
+The core principle of mxisd is to map between Matrix IDs and 3PIDs (Third-Party IDentifiers) for the Homeserver and its
+users. 3PIDs can be anything that uniquely and globally identify a user, like:
 - Email address
 - Phone number
-- Employee number
 - Skype/Live ID
 - Twitter handle
 - Facebook ID
-- ...
 
-mxisd is an enhanced Identity service, which implements the [Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html)
-but also several other features that greatly enhance user experience within Matrix.
-
-mxisd is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built as a
-single coherent product.
+If you are unfamiliar with the Identity vocabulary and concepts in Matrix, **please read this [introduction](docs/concepts.md)**.
 
 # Features
-As a [regular Matrix Identity service](docs/features/identity.md):
-- Search for people by 3PID using its own Identity stores (LDAP, SQL, etc.)
-- Invite people to rooms by 3PID using its own Identity stores, with [notifications](docs/README.md)
-to the invitee (Email, SMS, etc.)
+[Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://kamax.io/matrix/api/identity_service/unstable.html#general-principles):
+- Search for people by 3PID using its own Identity stores
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#association-lookup))
+- Invite people to rooms by 3PID using its own Identity stores, with notifications to the invitee (Email, SMS, etc.)
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#post-matrix-identity-api-v1-store-invite))
 - Allow users to add 3PIDs to their settings/profile
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#establishing-associations))
+- Register accounts on your Homeserver with 3PIDs
+  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#establishing-associations))
 
 As an enhanced Identity service:
-- Use a recursive lookup mechanism when searching and inviting people by 3PID, allowing to fetch data from:
-  - Own Identity store
+- [Federation](docs/features/federation.md): Use a recursive lookup mechanism when searching and inviting people by 3PID,
+  allowing to fetch data from:
+  - Own Identity store(s)
   - Federated Identity servers, if applicable to the 3PID
   - Arbitrary Identity servers
   - Central Matrix Identity servers
-- [Extensive control of where 3PIDs are transmited](docs/sessions/3pid.md), so they are not leaked publicly by users
-- [Authentication support](docs/features/authentication.md) for [synapse](https://github.com/matrix-org/synapse) via the
-[REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth)
-- [Directory integration](docs/features/directory-users.md) which allows you to search for users within your
-organisation, even without prior Matrix contact
-- [Auto-fill of user profile](docs/features/authentication.md) (Display name, 3PIDs) via the
-[REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth)
+- [Session Control](docs/threepids/session/session.md): Extensive control of where 3PIDs are transmitted so they are not
+  leaked publicly by users
+- [Authentication](docs/features/authentication.md): Use your Identity stores to perform authentication in [synapse](https://github.com/matrix-org/synapse)
+  via the [REST password provider](https://github.com/kamax-io/matrix-synapse-rest-auth)
+- [Directory search](docs/features/directory.md) which allows you to search for users within your organisation,
+  even without prior contact within Matrix using arbitrary search terms
+- [Auto-fill of user profile](docs/features/authentication.md#profile-auto-fill) (Display name, 3PIDs)
+- [Bridge Integration](docs/features/bridge-integration.md): Automatically bridge users without a published Matrix ID
 
-# Why use mxisd
-- Use your existing Identity store, do not duplicate information
+# Use cases
+- Use your existing Identity stores, do not duplicate your users information
 - Auto-fill user profiles with relevant information
-- As an organisation, stay in control of 3PIDs so they are not published to the central Matrix.org servers where they
-currently **cannot be removed**
+- As an organisation, stay in control of your data so it is not published to other servers by default where they
+  currently **cannot be removed**
 - Users can directly find each other using whatever attribute is relevant within your Identity store
-- Federate your Identity lookups so you can discover others and/or others can discover you, all with extensive ACLs
+- Federate your Identity server so you can discover others and/or others can discover you
 
-# Quick Start
-1. [Preparation](#preparation)
-2. [Install](#install)
-3. [Configure](#configure)
-4. [Integrate](#integrate)
-5. [Validate](#validate)
-
-Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
-talk to the central Matrix.org Identity service.  
-This will be a good ground work for further integration with your existing Identity stores.
-
-## Preparation
-You will need:
-- Homeserver
-- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
-
-As synapse requires an HTTPS connection when talking to an Identity service, a reverse proxy is required as mxisd does
-not support HTTPS listener at this time.
-
-For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.  
-You can also use a dedicated domain for mxisd, but will not have access to some features.
-
-Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-io/mxisd/wiki/Gotchas#nating) if you use the same
-hostname.
-
-The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
-If you would like a high-level view of the infrastructure and how each feature is integrated, see the
-[dedicated document](docs/architecture.md)
-
-## Install
-Install via:
-- [Debian package](docs/install/debian.md)
-- [Docker image](docs/install/docker.md)
-- [Sources](docs/build.md)
-
-See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
-
-## Configure
-Create/edit a minimal configuration (see installer doc for the location):
-```
-matrix.domain: 'MyMatrixDomain.org'
-key.path: '/path/to/signing.key.file'
-storage.provider.sqlite.database: '/path/to/mxisd.db'
-```  
-- `matrix.domain` should be set to your Homeserver domain
-- `key.path` will store the signing keys, which must be kept safe!
-- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
-
-If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
-Complete configuration guide is available [here](docs/configure.md).
-
-## Integrate
-For an overview of a typical mxisd infrastructure, see the [dedicated document](docs/architecture.md)
-### Reverse proxy
-#### Apache2
-In the VirtualHost handling the domain with SSL, add the following line and replace `0.0.0.0` by the right address/host.  
-**This line MUST be present before the one for the homeserver!**
-```
-ProxyPass /_matrix/identity/ http://0.0.0.0:8090/_matrix/identity/
-```
-
-Typical VirtualHost configuration would be:
-```
-<VirtualHost *:443>
-    ServerName example.org
-    
-    ...
-    
-    ProxyPreserveHost on
-    ProxyPass /_matrix/identity/ http://10.1.2.3:8090/_matrix/identity/
-    ProxyPass /_matrix/ http://10.1.2.3:8008/_matrix/
-</VirtualHost>
-```
-
-### Synapse
-Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
-In a typical configuration, you would end up with something similair to:
-```
-trusted_third_party_id_servers:
-    - matrix.org
-    - vector.im
-    - example.org
-```
-It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is allowed by synapse. 
-
-### Federation and network discovery
-See the [dedicated document](docs/features/federation.md).
-
-## Validate
-Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
-the relevant hostname which you configured in your reverse proxy.  
-Invite `mxisd-lookup-test@kamax.io` to a room, which should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.  
-**NOTE:** you might not see a Matrix suggestion for the e-mail address, which is normal. Still proceed with the invite.
-  
-If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
-If it did not work, [get in touch](#support) and we'll do our best to get you started.
-
-You can now integrate mxisd further with your infrastructure using the various [features](docs/README.md) guides.
+# Getting started
+See the [dedicated document](docs/getting-started.md)
 
 # Support
 ## Community
-If you need help, want to report a bug or just say hi, you can reach us on Matrix at 
-[#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) or [directly peek anonymously](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/).
-For more high-level discussion about the Identity Server architecture/API, go to 
-[#matrix-identity:matrix.org](https://matrix.to/#/#matrix-identity:matrix.org)
+Over Matrix: [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) ([Preview](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/))
 
-## Professional
-If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open source technologies/products, 
-please visit [our website](https://www.kamax.io/) to get in touch with us and get a quote.
+For more high-level discussion about the Identity Server architecture/API, go to  [#matrix-identity:kamax.io](https://matrix.to/#/#matrix-identity:kamax.io)
 
-We offer affordable monthly/yearly support plans for mxisd, synapse or your full Matrix infrastructure.
+## Commercial
+If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open
+source technologies/products:
+- Visit our [website](https://www.kamax.io/) to get in touch with us and get a quote.
+- Come in our general Matrix room: [#kamax-matrix:kamax.io](https://matrix.to/#/#kamax-matrix:kamax.io)
 
 # Contribute 
-First and foremost, the best way to contribute is to use mxisd and tell us about it!  
-We would love to hear about your experience and get your feedback on how to make it an awesome product. 
-
 You can contribute as a community member by:
+- Giving us feedback about your usage of mxisd, even if it seems unimportant or if all is working well!
 - Opening issues for any weird behaviour or bug. mxisd should feel natural, let us know if it does not!
 - Helping us improve the documentation: tell us what is good or not good (in an issue or in Matrix), or make a PR with
 changes you feel improve the doc.
 - Contribute code directly: we love contributors! All your contributions will be licensed under AGPLv3.
-- Donate! any donation is welcome, regardless how small or big. This will directly be used for the fixed costs and
-developer time.
+- [Donate!](https://liberapay.com/maximusdor/) Any donation is welcome, regardless how small or big, and will directly
+be used for the fixed costs and developer time of mxisd.
 
 You can contribute as an organisation/corporation by:
-- Get a [support contract](#support-professional). This is the best way you can help us as it ensures mxisd is
+- Get a [support contract](#commercial). This is the best way you can help us as it ensures mxisd is
 maintained regularly and you get direct access to the support team.
 - Sponsoring new features or bug fixes. [Get in touch](#contact) so we can discuss it further.
 
 # FAQ
-### Do I need to use mxisd if I run a Homeserver?
-No, but it is recommended, even if you don't use any backends or integration.
-
-mxisd in its default configuration will use federation and involve the central Matrix.org Identity servers when
-performing queries, giving you access to at least the same information as if you were not running it.
-
-It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
-privacy consequences, which is not the case with the central Matrix.org servers.
-
-So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
-simple features on top of it.
-
-### I already use the synapse LDAP3 auth provider, why should I care about mxisd?
-The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) only handles on specific flow:
-validate credentials at login.
-
-It does not:
-- Auto-provision user profiles
-- Integrate with Identity management
-- Integrate with Directory searches
-- Protect you against the username case sensitivites issues in synapse
-
-mxisd is a replacement and enhancement of it, and also offers coherent results in all areas, which LDAP3 auth provider
-does not.
-
-### I saw that sydent is the official Identity server implemenation of the Matrix team, I should use that!
-You can, but [sydent](https://github.com/matrix-org/sydent):
-- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
-- is not meant to be linked to a specific Homeserver / domain
-- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
-- forces you to duplicate all your identity data, so people can be found by 3PIDs
-- forces users to enter all their emails and phone numbers manually in their profile
-
-So really, you should go with mxisd.
-
-### I'm not sure I understand what an "Identity server" is supposed to be or do
-The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
-what they want to do with that part of the ecosystem. Therefore, "Identity" is a misleading word currently.  
-Given the scope of the current Identity Service API, it would be best called "Invitation service".
-
-Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
-for groups/corporations/organisation. This is where mxisd comes in.
-
-mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
-up to its "Identity server" name.
-
-### So mxisd is just a big hack! I don't want to use non-official features!
-mxisd primary concern is to always be compatible with the Matrix ecosystem and the Identity service API.  
-Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
-
-We also directly talk with the Matrix developers to ensure all features we implement have their approval, and that we
-are in line with their vision of Identity management within the Matrix ecosystem.
-
-Therefore, using mxisd is a safe choice. It will be like using the central Matrix.org Identity servers, yet not closing
-the door to very nice enhancements and integrations.
-
-### Should I use mxisd if I don't host my own Homeserver?
-No
+See the [dedicated document](docs/faq.md)
 
 # Contact
 Get in touch via:
-- Matrix at [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io)
-- Email, see our website: [Kamax.io](https://www.kamax.io)
+- Matrix: [#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io)
+- Email: see our website: [Kamax.io](https://www.kamax.io)

application.example.yaml

@@ -6,6 +6,7 @@
 # Matrix config items #
 #######################
 # Matrix domain, same as the domain configure in your Homeserver configuration.
+# (note: in Synapse Homeserver, the Matrix domain may be defined as 'server_name' in configuration file).
 #
 # This is used to build the various identifiers for identity, auth and directory.
 matrix.domain: ''
@@ -17,10 +18,12 @@ matrix.domain: ''
 # Absolute path for the Identity Server signing key.
 # During testing, /var/tmp/mxisd.key is a possible value
 #
-# For production, use a stable location like:
+# For production, recommended location shall be one of the following:
 #   - /var/opt/mxisd/sign.key
 #   - /var/local/mxisd/sign.key
 #   - /var/lib/mxisd/sign.key
+#
+# The signing key is auto-generated during execution time if not present.
 key.path: ''
 
 
@@ -92,5 +95,5 @@ threepid.medium.email.connectors.smtp.login: "matrix-identity@example.org"
 # Password for the account
 threepid.medium.email.connectors.smtp.password: "ThePassword"
 
-# The e-mail to send as. If empty, will be the same as login
+# The e-mail to send as.
 threepid.medium.email.identity.from: "matrix-identity@example.org"

build.gradle

@@ -47,7 +47,7 @@ String gitVersion() {
     def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")
     ByteArrayOutputStream out = new ByteArrayOutputStream()
     exec {
-        commandLine = ['git', 'describe', '--always', '--dirty']
+        commandLine = ['git', 'describe', '--tags', '--always', '--dirty']
         standardOutput = out
     }
     def v = out.toString().replace(System.lineSeparator(), '')
@@ -66,6 +66,7 @@ buildscript {
 
 repositories {
     maven { url "https://kamax.io/maven/releases/" }
+    maven { url "https://kamax.io/maven/snapshots/" }
     mavenCentral()
 }
 
@@ -74,13 +75,13 @@ dependencies {
     compile 'commons-io:commons-io:2.5'
 
     // Spring Boot - standalone app
-    compile 'org.springframework.boot:spring-boot-starter-web:1.5.3.RELEASE'
+    compile 'org.springframework.boot:spring-boot-starter-web:1.5.10.RELEASE'
 
     // Thymeleaf for HTML templates
-    compile "org.springframework.boot:spring-boot-starter-thymeleaf:1.5.3.RELEASE"
+    compile "org.springframework.boot:spring-boot-starter-thymeleaf:1.5.10.RELEASE"
 
     // Matrix Java SDK
-    compile 'io.kamax:matrix-java-sdk:0.0.2'
+    compile 'io.kamax:matrix-java-sdk:0.0.11'
 
     // ed25519 handling
     compile 'net.i2p.crypto:eddsa:0.1.0'
@@ -94,9 +95,6 @@ dependencies {
     // HTTP connections
     compile 'org.apache.httpcomponents:httpclient:4.5.3'
 
-    // JSON
-    compile 'com.google.code.gson:gson:2.8.1'
-
     // Phone numbers validation
     compile 'com.googlecode.libphonenumber:libphonenumber:8.7.1'
 
@@ -104,6 +102,9 @@ dependencies {
     compile 'com.sun.mail:javax.mail:1.5.6'
     compile 'javax.mail:javax.mail-api:1.5.6'
 
+    // Google Client APIs
+    compile 'com.google.api-client:google-api-client:1.23.0'
+
     // Google Firebase Authentication backend
     compile 'com.google.firebase:firebase-admin:5.3.0'
 
@@ -119,6 +120,9 @@ dependencies {
     // PostgreSQL
     compile 'org.postgresql:postgresql:42.1.4'
 
+    // MariaDB/MySQL
+    compile 'org.mariadb.jdbc:mariadb-java-client:2.1.2'
+
     // Twilio SDK for SMS
     compile 'com.twilio.sdk:twilio:7.14.5'
 

docs/README.md

@@ -1,25 +1,22 @@
 # Table of Contents
-
+- [Build from sources](build.md) (Optional)
 - Installation
   - [Debian package](install/debian.md)
+  - [ArchLinux](install/archlinux.md)
   - [Docker](install/docker.md)
-- [Build from source](build.md)
+  - [From source](install/source.md)
 - [Architecture overview](architecture.md)
 - [Configuration](configure.md)
 - Features
-  - [Matrix Identity Service](features/identity.md)
-  - [Homeserver Authentication](features/authentication.md)
-  - [Directory seach](features/directory-users.md)
-  - [Identity server Federation](features/federation.md)
+  - [Authentication](features/authentication.md)
+  - [Directory search](features/directory.md)
+  - [Identity](features/identity.md)
+  - [Federation](features/federation.md)
   - [Bridge integration](features/bridge-integration.md)
-- Backends
-  - [LDAP](backends/ldap.md)
-  - [SQL](backends/sql.md)
-  - [REST](backends/rest.md)
-  - [Google Firebase](backends/firebase.md)
+- [Identity Stores](stores/README.md)
 - Notifications
   - Handlers
-    - [Basic](threepids/notifications/basic-handler.md)
-    - [SendGrid](threepids/notifications/sendgrid-handler.md)
-- [Sessions](sessions/3pid.md)
-  - [Views](sessions/3pid-views.md)
+    - [Basic](threepids/notification/basic-handler.md)
+    - [SendGrid](threepids/notification/sendgrid-handler.md)
+- [Sessions](threepids/session/session.md)
+  - [Views](threepids/session/session-views.md)

docs/_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-cayman

docs/architecture.md

@@ -1,6 +1,6 @@
 # Architecture
 ## Overview
-### Basic setup without integration or federation
+### Basic setup with default settings
 ```
  Client
    |
@@ -14,17 +14,17 @@ TCP 443
        +--|------------------+            +---|-----------------------+
           |                                   |
           +<---------------------------------<+
-          |                                          Backends
-          |   +-------------------+                  +------+    +--------+
- TCP 8090 +-> | mxisd             |          +-----> | LDAP | -> | SQL DB |
-              |                   |          |       +------+    +--------+ ....
-              | - Profile's 3PIDs >----+     |
-              | - 3PID Invites    |    |     |
-              +-|-----------------+    +>----+
-                |                      |     |       +--------------------------+
-                |                      |     |       | Central Identity service |
-                +>-------------------->+     +-----> | Matrix.org / Vector.im   |
-                |                            TCP 443 +--------------------------+
+          |
+          |   +-------------------+
+ TCP 8090 +-> | mxisd             |
+              |                   |
+              | - Profile's 3PIDs >----+
+              | - 3PID Invites    |    |             +--------------------------+
+              +-|-----------------+    +>----------> | Central Identity service |
+                |                      |   TCP 443   | Matrix.org / Vector.im   |
+                |                      |             +--------------------------+
+                +>-------------------->+
+                |
              TCP 443
                 |  +------------------------+
                 |  | Remote Federated       |
@@ -37,7 +37,7 @@ TCP 443
 See the [dedicated document](features/authentication.md).
 
 ### With Directory
-See the [dedicated document](features/directory-users.md).
+See the [dedicated document](features/directory.md).
 
 ### With Federation
 See the [dedicated document](features/federation.md).

docs/backends/firebase.md

@@ -1,9 +0,0 @@
-# Google Firebase
-## Configuration
-To be completed. For now, see default structure and values:
-```
-firebase:
-  enabled: false
-  credentials: '/path/to/firebase/credentials.json'
-  database: 'https://my-project.firebaseio.com/'
-```

docs/backends/ldap.md

@@ -1,86 +0,0 @@
-# AD/Samba/LDAP backend
-## Configuration
-### Structure and default values
-```
-ldap:
-  enabled: false
-  filter: ''
-  connection:
-    host: ''
-    tls: false
-    port: 389
-    bindDn: ''
-    bindPassword: ''
-    baseDn: ''
-  attribute:
-    uid:
-      type: 'uid'
-      value: 'userPrincipalName'
-    name: 'displayName'
-    threepid:
-      email:
-        - 'mailPrimaryAddress'
-        - 'mail'
-        - 'otherMailbox'
-      msisdn:
-        - 'telephoneNumber'
-        - 'mobile'
-        - 'homePhone'
-        - 'otherTelephone'
-        - 'otherMobile'
-        - 'otherHomePhone'
-  auth:
-    filter: ''
-  directory:
-    attribute:
-      other: []
-    filter: ''
-  identity:
-    filter: ''
-    medium:
-      email: ''
-      msisdn: ''
-```
-### General
-| Item      | Description                                                                               |
-|-----------|-------------------------------------------------------------------------------------------|
-| `enabled` | Globaly enable/disable the LDAP backend                                                   |
-| `filter`  | Global filter to apply on all LDAP queries. Can be overwritten in each applicable section |
-
-### Connection
-| Item           | Description                                          |
-|----------------|------------------------------------------------------|
-| `host`         | Host to connect to                                   |
-| `port`         | Port to use                                          |
-| `tls`          | boolean to use TLS or not (STARTLS is not supported) |
-| `bindDn`       | Bind DN for authentication                           |
-| `bindPassword` | Bind password                                        |
-| `baseDn`       | Base DN for queries                                  |
-
-### Attributes
-| Item        | Description                                                                                                            |
-|-------------|------------------------------------------------------------------------------------------------------------------------|
-| `uid.type`  | Indicate how to process the User ID (UID) attribute:                                                                   |
-|             |   - `uid` will consider the value as the [Localpart](https://matrix.org/docs/spec/intro.html#user-identifiers)         |
-|             |   - `mxid` will consider the value as a complete [Matrix ID](https://matrix.org/docs/spec/intro.html#user-identifiers) |
-| `uid.value` | Attribute name refering to the User ID. This is typically `userPrincipalName` on AD/Samba setups and `uid` in LDAP     |
-| `name`      | Attribute name that contains the [Display Name](https://matrix.org/docs/spec/intro.html#profiles) of the user          |
-| `threepid`  | Namespace where each key is a 3PID type and contains a list of attributes                                              |
-
-### Authentication
-| Item     | Description                                                                                      |
-|----------|--------------------------------------------------------------------------------------------------|
-| `filter` | Specific user filter applied during authentication. Global filter is used if empty/blank/not set |
-
-### Directory
-| Item              | Description                                                         |
-|-------------------|---------------------------------------------------------------------|
-| `attribute.other` | Additional attributes to be used when performing directory searches |
-| `filter`          | Specific user filter applied during directory search.               |
-|                   | Global filter is used if empty/blank/not set                        |
-
-### Identity
-| Item     | Description                                                                                       |
-|----------|---------------------------------------------------------------------------------------------------|
-| `filter` | Specific user filter applied during identity search. Global filter is used if empty/blank/not set | 
-| `medium` | Namespace to overwrite generated queries from the list of attributes for each 3PID medium         |

docs/backends/sql.md

@@ -1,23 +0,0 @@
-# SQL Backend
-## Configuration
-To be completed. For now, see default structure and values:
-```
-sql:
-  enabled: false
-  type: 'sqlite' or 'postgresql'
-  connection: ''
-  auth:
-    enabled: false
-  directory:
-    enabled: false
-    query:
-      name:
-        type: 'localpart'
-        value: 'SELECT 1'
-      threepid:
-        type: 'localpart'
-        value: 'SELECT 1'
-  identity:
-    type: 'localpart'
-    query: 'SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?'
-```

docs/build.md

@@ -1,14 +1,17 @@
 # From source
 - [Binaries](#binaries)
+  - [Requirements](#requirements)
+  - [Build](#build)
 - [Debian package](#debian-package)
 - [Docker image](#docker-image)
+- [Next steps](#next-steps)
 
 ## Binaries
 ### Requirements
 - JDK 1.8
 
 ### Build
-```
+```bash
 git clone https://github.com/kamax-io/mxisd.git
 cd mxisd
 ./gradlew build
@@ -16,90 +19,55 @@ cd mxisd
 
 Create a new configuration file by coping `application.example.yaml` to `application.yaml` and edit to your needs.  
 For advanced configuration, see the [Configure section](configure.md).  
+**NOTE**: `application.yaml` is also called `mxisd.yaml` in some specific installations.
 
 Start the server in foreground to validate the build and configuration:
-```
+```bash
 java -jar build/libs/mxisd.jar
 ```
 
 Ensure the signing key is available:
-```
+```bash
 $ curl 'http://localhost:8090/_matrix/identity/api/v1/pubkey/ed25519:0'
 {"public_key":"..."}
 ```
 
 Test basic recursive lookup (requires Internet connection with access to TCP 443):
-```
-$ curl 'http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=mxisd-lookup-test@kamax.io'
-{"address":"mxisd-lookup-test@kamax.io","medium":"email","mxid":"@mxisd-lookup-test:kamax.io",...}
+```bash
+$ curl 'http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=mxisd-federation-test@kamax.io'
+{"address":"mxisd-federation-test@kamax.io","medium":"email","mxid":"@mxisd-lookup-test:kamax.io",...}
 ```
 
 If you enabled LDAP, you can also validate your config with a similar request after replacing the `address` value with
 something present within your LDAP
-```
+```bash
 curl 'http://localhost:8090/_matrix/identity/api/v1/lookup?medium=email&address=john.doe@example.org'
 ```
 
 If you plan on testing the integration with a homeserver, you will need to run an HTTPS reverse proxy in front of it
 as the reference Home Server implementation [synapse](https://github.com/matrix-org/synapse) requires a HTTPS connection
 to an ID server.  
-See the [Integration section](#integration) for more details.
 
-### Install
-1. Prepare files and directories:
-```
-# Create a dedicated user
-useradd -r mxisd
-
-# Create bin directory
-mkdir /opt/mxisd
-
-# Create config directory and set ownership
-mkdir /etc/opt/mxisd
-chown mxisd /etc/opt/mxisd
-
-# Create data directory and set ownership
-mkdir /var/opt/mxisd
-chown mxisd /var/opt/mxisd
-
-# Copy <repo root>/build/libs/mxisd.jar to bin directory
-cp ./build/libs/mxisd.jar /opt/mxisd/
-chown mxisd /opt/mxisd/mxisd.jar
-chmod a+x /opt/mxisd/mxisd.jar
-
-# Create symlink for easy exec
-ln -s /opt/mxisd/mxisd.jar /usr/bin/mxisd
-```
-
-2. Copy the sample config file `./application.example.yaml` to `/etc/opt/mxisd/mxisd.yaml`, edit to your needs
-4. Copy `<repo root>/src/systemd/mxisd.service` to `/etc/systemd/system/` and edit if needed
-5. Enable service for auto-startup
-```
-systemctl enable mxisd
-```
-6. Start mxisd
-```
-systemctl start mxisd
-```
+Next step: [Install your compiled binaries](install/source.md)
 
 ## Debian package
-### Requirements
+Requirements:
 - fakeroot
 - dpkg-deb
 
-### Build
 [Build mxisd](#build) then:
-```
+```bash
 ./gradlew buildDeb 
 ```
-You will find the debian package in `build/dist`
+You will find the debian package in `build/dist`.  
+Then follow the instruction in the [Debian package](install/debian.md) document.
 
 ## Docker image
 [Build mxisd](#build) then:
-```
+```bash
 ./gradlew dockerBuild
 ```
-You can run a container of the given image and test it with the following command (adapt volumes host paths):
-```
-docker run -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd:latest-dev
-```
+Then follow the instructions in the [Docker install](install/docker.md#configure) document.
+
+## Next steps
+- [Integrate with your infrastructure](getting-started.md#integrate)

docs/concepts.md

@@ -0,0 +1,43 @@
+# Concepts
+- [Matrix](#matrix)
+- [mxisd](#mxisd)
+
+## Matrix
+The following concepts are part of the Matrix ecosystem and specification.
+
+### 3PID
+`3PID` stands for Third-Party Identifier.  
+It is also commonly written:
+- `3pid`
+- `tpid`
+
+A 3PID is a globally unique canonical identifier which is made of:
+- Medium, which describes what network it belongs to (Email, Phone, Twitter, Discord, etc.)
+- Address, the actual value people typically use on a daily basis.
+
+mxisd core mission is to map those identifiers to Matrix User IDs.
+
+### Homeserver
+Where a user **account and data** are stored.
+
+### Identity server
+An Identity server:
+- Does lookup of 3PIDs to User Matrix IDs.
+- Does validate 3PIDs ownership, typically by sending a code that the user has to enter in an application/on a website.
+- Does send notifications about room invites where no Matrix User ID could be found for the invitee.
+
+An Identity server:
+- **DOES NOT** store user accounts.
+- **DOES NOT** store user data.
+- **DOES NOT** allow migration of user account and/or data between homeservers. 
+
+### 3PID session
+The fact to validate a 3PID (email, phone number, etc.) via the introduction of a token which was sent to the 3PID address.
+
+## mxisd
+The following concepts are specific to mxisd.
+
+### Identity store
+Where your user accounts and 3PID mappings are stored.
+
+mxisd itself **DOES NOT STORE** user accounts or 3PID mappings.

docs/configure.md

@@ -1,18 +1,25 @@
 # Configuration
-- [Syntax](#syntax)
-- [Variables](#variables)
-- [Categories](#categories)
+- [Concepts](#concepts)
+  - [Syntax](#syntax)
+  - [Variables](#variables)
+- [Matrix](#matrix)
+- [Server](#server)
+- [Storage](#storage)
+- [Identity stores](#identity-stores)
+- [3PID Validation sessions](#3pid-validation-sessions)
+- [Notifications](#notifications)
 
-## Syntax
-The configuration file is YAML based, allowing two types of syntax.
+## Concepts
+### Syntax
+The configuration file is [YAML](http://yaml.org/) based, allowing two types of syntax.
 
 Properties-like:
-```
+```yaml
 my.config.item: 'value'
 ```
 
 Object-like:
-```
+```yaml
 my:
   config:
     item: 'value'
@@ -21,12 +28,10 @@ my:
 These can also be combined within the same file.  
 Both syntax will be used interchangeably in these documents.
 
-Default values for each possible option are documented [here](../src/main/resources/application.yaml)
-
-## Variables
+### Variables
 It is possible to copy the value of a configuration item into another using the syntax `${config.key.item}`.  
 Example that will copy the value of `matrix.domain` into `server.name`:
-```
+```yaml
 matrix:
   domain: 'example.org'
 
@@ -36,148 +41,62 @@ server:
 
 **WARNING:** mxisd might overwrite/adapt some values during launch. Those changes will not be reflected into copied keys.
 
-## Categories
-For each category below, the base configuration path will be given, which needs to be appened to every configuration
-item described.
 
-Example: if the base path was `basePath` and the following table was given:
-
-| Name | Purpose |
-|------|---------|
-| item1 | To give an example |
-| item2 | To give another example |
-
-The following configurations could be used, all being equivalent:
-```
-basePath.item1: 'myValue'
-basePath.item2: 'myOtherValue'
-```
-```
-basePath:
-  item1: 'myValue'
-  item2: 'myOtherValue'
-```
-```
-basePath.item1: 'myValue'
-basePath:
-  item2: 'myOtherValue'
-```
+## Matrix
+`matrix.domain`
+Matrix domain name, same as the Homeserver, used to build appropriate Matrix IDs |
 
 ---
 
-In case a relative base path is given, it is appended to the one above.
-
-Example: With base path `basePath`, the relative base `relativeBasePath` and the following table:
-  
-| Name | Purpose |
-|------|---------|
-| item1 | To give an example |
-| item2 | To give another example |
-
-The following configurations could be used, all being equivalent:
-```
-basePath.relativeBasePath.item1: 'myValue'
-basePath.relativeBasePath.item2: 'myOtherValue'
-```
-```
-basePath:
-  relativeBasePath:
-    item1: 'myValue'
-    item2: 'myOtherValue'
-```
-```
-basePath.relativeBasePath.item1: 'myValue'
-basePath:
-  relativeBasePath:
-    item2: 'myOtherValue'
-```
-
-### Matrix
-Base path: `matrix`
-
-| Name | Purpose |
-|------|---------|
-| `domain` | Matrix domain name, same as the Homeserver, used to build appropriate Matrix IDs |
-
----
-
-Relative base path: `identity`
-
-| Name | Purpose |
-|------|---------|
-| `servers` | Namespace to create arbitrary list of Identity servers, usable in other parts of the configuration |
+`matrix.identity.servers`
+Namespace to create arbitrary list of Identity servers, usable in other parts of the configuration |
 
 Example:
-```
+```yaml
 matrix.identity.servers:
-  root:
-    - 'https://matrix.org'
+  myOtherServers:
+    - 'https://other1.example.org'
+    - 'https://other2.example.org'
 ```
 Create a list under the label `root` containing a single Identity server, `https://matrix.org`
-### Server
-| Name | Purpose |
-|------|---------|
-| `name` | Public hostname of mxisd, if different from the Matrix domain |
-| `port` | HTTP port to listen on (unencrypted) |
-| `publicUrl` | Defaults to `https://${server.name}` |
 
-### Storage
-Base path: `storage`
+## Server
+- `server.name`: Public hostname of mxisd, if different from the Matrix domain.
+- `server.port`: HTTP port to listen on (unencrypted)
+- `server.publicUrl`: Defaults to `https://${server.name}`
 
-| Name | Purpose |
-|------|---------|
-| `backend` | Specify which SQL backend to use. only `sqlite` is currently supported. |
+## Storage
+### SQLite
+`storage.provider.sqlite.database`: Absolute location of the SQLite database
 
----
-Relative base path: `provider.sqlite`
+## Identity stores
+See the [Identity stores](stores/README.md) for specific configuration
 
-| Name | Purpose |
-|------|---------|
-| `database` | Absolute location of the SQLite database |
+## 3PID Validation sessions
+See the dedicated documents:
+- [Flow](threepids/session/session.md)
+- [Branding](threepids/session/session-views.md)
 
-### Backends
-- [LDAP](backends/ldap.md)
-- [SQL](backends/sql.md)
-- [REST](backends/rest.md)
-- [Google Firebase](backends/firebase.md)
-
-### Lookups
-work in progress, should not be configured.
-
-### Sessions
-See the [dedicated document](sessions/3pid.md)
-
-### Notifications
-Base path: `notification`
-
-| Name | Purpose |
-|------|---------|
-| handler | Namespace to specify the handler to use for each 3PID |
-| handlers | Namespace used by individual handlers for their own configuration |
+## Notifications
+- `notification.handler.<3PID medium>`: Handler to use for the given 3PID medium. Repeatable.
 
 Example:
+```yaml
+notification.handler.email: 'sendgrid'
+notification.handler.msisdn: 'raw'
 ```
-notification:
-  handler:
-    email: 'sendgrid'
-    msisdn: 'raw'
-  handlers:
-    raw:
-      ...
-    sendgrid:
-      ...
-```
-- Emails notifications would use the `sendgrid` handler, which define its own configuration user `handlers.sendgrid`
+- Emails notifications would use the `sendgrid` handler, which define its own configuration under `notification.handlers.sendgrid`
 - Phone notification would use the `raw` handler, basic default built-in handler of mxisd
-#### Handlers
-Relative base path: `handlers`
+
+### Handlers
+- `notification.handers.<handler ID>`: Handler-specific configuration for the given handler ID. Repeatable.
+
+Example:
+```yaml
+notification.handlers.raw: ...
+notification.handlers.sendgrid: ...
+```
 
 Built-in:
-- [Basic](threepids/notifications/basic-handler.md)
-- [SendGrid](threepids/notifications/sendgrid-handler.md)
-
-### Views
-See the [dedicated document](sessions/3pid-views.md)
-
-### DNS Overwite
-Specific to other features.
+- [Raw](threepids/notification/basic-handler.md)
+- [SendGrid](threepids/notification/sendgrid-handler.md)

docs/faq.md

@@ -0,0 +1,89 @@
+# Frequently Asked Questions
+### This is all very complicated and I'm getting confused with all the words, concepts and diagrams - Help!
+Matrix is still a very young protocol and there are a whole lot of rough edges.  
+Identity in Matrix is one of the most difficult topic, mainly as it has not received much love in the past years.
+
+We have tried our best to put together documentation that requires almost no knowledge of Matrix inner workings to get a
+first basic setup running which relies on you reading the documentation in the right order:
+- [The  Concepts](concepts.md) in few words.
+- [Getting Started](getting-started.md) step-by-step to a minimal working install.
+- [Identity stores](stores/README.md) you wish to fetch data from.
+- [Features](features) you are interested in that will use your Identity store(s) data.
+
+**IMPORTANT**: Be aware that mxisd tries to fit within the current protocol and existing products and basic understanding
+of the Matrix protocol is required for some advanced features.
+
+If all fails, come over to [the project room](https://matrix.to/#/#mxisd:kamax.io) and we'll do our best to get you
+started and answer questions you might have.
+
+### Do I need to use mxisd if I run a Homeserver?
+No, but it is strongly recommended, even if you don't use any Identity store or integration.
+
+In its default configuration, mxisd will talk to the central Matrix Identity servers and use other federated public
+servers when performing queries, giving you access to at least the same information as if you were not running it.
+
+It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
+privacy consequences, which is not the case with the central Matrix.org servers.
+
+So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
+simple features on top of it.
+
+### I'm not sure I understand what an "Identity server" is supposed to be or do...
+The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
+what they want to do with that part of the ecosystem. Therefore, "Identity" is currently a misleading word and concept.
+Given the scope of the current Identity Service API, it would be best called "Invitation service".
+
+Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
+for groups/corporations/organisation. This is where mxisd comes in.
+
+mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
+up to its "Identity server" name.
+
+### Can I migrate my existing account on another Matrix server with mxisd?
+No.
+
+Accounts cannot currently migrate/move from one server to another.  
+See a [brief explanation document](concepts.md) about Matrix and mxisd concepts and vocabulary.
+
+### I already use the synapse LDAP3 auth provider. Why should I care about mxisd?
+The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) is not longer maintained and
+only handles on specific flow: validate credentials at login.
+
+It does not:
+- Auto-provision user profiles
+- Integrate with Identity management
+- Integrate with Directory searches
+
+mxisd is a replacement and enhancement of it, offering coherent results in all areas, which the LDAP3 auth provider
+does not.
+
+### Sydent is the official Identity server implementation of the Matrix team. Why not use that?
+You can, but [sydent](https://github.com/matrix-org/sydent):
+- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
+- is not meant to be linked to a specific Homeserver / domain
+- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
+- forces you to duplicate all your identity data, so people can be found by 3PIDs
+- forces users to enter all their emails and phone numbers manually in their profile
+
+So really, you should go with mxisd.
+
+### Will I loose access to the central Matrix.org/Vector.im Identity data if I use mxisd?
+No.
+
+In its default configuration, mxisd act as a proxy to Matrix.org/Vector.im. You will have access to the same data and
+behaviour than if you were using them directly. There is no downside in using mxisd with the default configuration.
+
+mxisd can also be configured not to talk to the central Identity servers if you wish.
+
+### So mxisd is just a big hack! I don't want to use non-official features!
+mxisd primary concern is to always be compatible with the Matrix ecosystem and the Identity service API.  
+Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
+
+Therefore, using mxisd is a safe choice. It will be like using the central Matrix.org Identity servers, yet not closing
+the door to a growing list of enhancements and integrations.
+
+### Should I use mxisd if I don't host my own Homeserver?
+No.
+
+It is possible, but it is not supported and the scope of features will be extremely limited.
+Please consider hosting your own Homeserver and using mxisd alongside it.

docs/features/authentication.md

@@ -1,28 +1,154 @@
 # Authentication
-Performed via [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)  
-Point the `endpoint` to mxisd internal IP on port 8090
+- [Description](#description)
+- [Basic](#basic)
+  - [Overview](#overview)
+  - [synapse](#synapse)
+  - [mxisd](#mxisd)
+  - [Validate](#validate)
+  - [Next steps](#next-steps)
+    - [Profile auto-fil](#profile-auto-fill)
+- [Advanced](#advanced)
+  - [Overview](#overview-1)
+  - [Requirements](#requirements)
+  - [Configuration](#configuration)
+    - [Reverse Proxy](#reverse-proxy)
+      - [Apache2](#apache2)
+    - [DNS Overwrite](#dns-overwrite)
 
-## Overview
+## Description
+Authentication is an enhanced feature of mxisd to ensure coherent and centralized identity management.  
+It allows to use Identity stores configured in mxisd to authenticate users on your Homeserver.
+
+Authentication is divided into two parts:
+- [Basic](#basic): authenticate with a regular username.
+- [Advanced](#advanced): same as basic with extra ability to authenticate using a 3PID.
+
+## Basic
+Authentication by username is possible by linking synapse and mxisd together using a specific module for synapse, also
+known as password provider.
+
+### Overview
+An overview of the Basic Authentication process:
 ```
-                                                                                    Backends
+                                                                                    Identity stores
  Client                                                                             +------+
    |                                            +-------------------------+    +--> | LDAP |
    |   +---------------+  /_matrix/identity     | mxisd                   |    |    +------+
    +-> | Reverse proxy | >------------------+   |                         |    |
        +--|------------+                    |   |                         |    |    +--------+
-          |                                 +-----> Check wiht backends >------+--> | SQL DB |
+          |                                 +-----> Check ID stores     >------+--> | SQL DB |
      Login request                          |   |                         |    |    +--------+
           |                                 |   |     |                   |    |
-          |   +--------------------------+  |   +-----|-------------------+    +-->  Others
+          |   +--------------------------+  |   +-----|-------------------+    +-->  ...
           +-> | Homeserver               |  |         |
               |                          |  |         |
               | - Validate credentials >----+         |
               |   Using REST auth module |            |
               |                          |            |
               | - Auto-provision <-------------------<+
-              |   user profiles          |    If valid credentials and supported by backend
+              |   user profiles          |    If valid credentials and supported by Identity store(s)
               +--------------------------+
 ```
+Performed on [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)
 
-## Profile auto-fill
-To be documented
+### Synapse
+- Install the [password provider](https://github.com/kamax-io/matrix-synapse-rest-auth)
+- Edit your **synapse** configuration:
+  - As described by the auth module documentation
+  - Set `endpoint` to `http://mxisdAddress:8090` - Replace `mxisdAddress` by an IP/host name that provides a direct
+  connection to mxisd.  
+  This **MUST NOT** be a public address, and SHOULD NOT go through a reverse proxy.
+- Restart synapse
+
+### mxisd
+- Configure and enable at least one [Identity store](../stores/README.md)
+- Restart mxisd
+
+### Validate
+Login on the Homeserver using credentials present in one of your Identity stores.
+
+## Next steps
+### Profile auto-fill
+Auto-filling user profile depends on its support by your configured Identity stores.  
+See your Identity store [documentation](../stores/README.md) on how to enable the feature.
+
+
+## Advanced
+The Authentication feature allows users to login to their Homeserver by using their 3PIDs in a configured Identity store.
+
+### Overview
+This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/login` as depicted below:
+```
+            +----------------------------+
+            |  Reverse Proxy             |
+            |                            |
+            |                            |     Step 1    +---------------------------+     Step 2
+            |                            |               |                           |
+Client+---->| /_matrix/client/r0/login +---------------->|                           | Look up address  +---------+
+            |                      ^     |               |  mxisd - Identity server  +----------------->| Backend |
+            |                      |     |               |                           |                  +---------+
+            | /_matrix/* +--+      +---------------------+                           |
+            |               |            |               +---------------+-----------+
+            |               |            |     Step 4                    |
+            |               |            |                               | Step 3
+            +---------------|------------+                               |
+                            |                                            | /_matrix/client/r0/login
+                            |                       +--------------+     |
+                            |                       |              |     |
+                            +---------------------->|  Homeserver  |<----+
+                                                    |              |
+                                                    +--------------+
+
+```
+
+Steps of user authentication using a 3PID:
+1. The intercepted login request is directly sent to mxisd instead of the Homeserver.
+2. Identity stores are queried for a matching user identity in order to modify the request to use the user name.
+3. The Homeserver, from which the request was intercepted, is queried using the request at previous step.
+   Its address is resolved using the DNS Overwrite feature to reach its internal address on a non-encrypted port.
+4. The response from the Homeserver is sent back to the client, believing it was the HS which directly answered.
+
+### Requirements
+- [Basic Authentication configured and working](#basic)
+- Reverse proxy setup
+- Homeserver
+- Compatible [Identity store](../stores/README.md)
+
+### Configuration
+#### Reverse Proxy
+##### Apache2
+The specific configuration to put under the relevant `VirtualHost`:
+```apache
+ProxyPass /_matrix/client/r0/login http://localhost:8090/_matrix/client/r0/login
+```
+`ProxyPreserveHost` or equivalent **must** be enabled to detect to which Homeserver mxisd should talk to when building results.
+
+Your VirtualHost should now look similar to:
+```apache
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/client/r0/login http://localhost:8090/_matrix/client/r0/login
+    ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
+    ProxyPass /_matrix http://localhost:8008/_matrix
+</VirtualHost>
+```
+
+#### DNS Overwrite
+Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
+the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
+
+To do so, put the following configuration in your mxisd configuration:
+```yaml
+dns.overwrite.homeserver.client:
+  - name: 'example.org'
+    value: 'http://localhost:8008'
+```
+`name` must be the hostname of the URL that clients use when connecting to the Homeserver.
+In case the hostname is the same as your Matrix domain, you can use `${matrix.domain}` to auto-populate the `value`
+using the `matrix.domain` configuration option and avoid duplicating it.
+
+`value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.

docs/features/bridge-integration.md

@@ -1 +1,28 @@
-To be documented
+# Bridge Integration
+To help natural bridge integration into the regular usage of a Matrix client, mxisd provides a way for bridge to reply
+to 3PID queries if no mapping was found, allowing seamless bridging to a network.
+
+This is performed by implementing a specific endpoint on the bridge to map a 3PID lookup to a virtual user.
+
+**NOTE**: This document is incomplete and might be misleading. In doubt, come in our Matrix room.  
+You can also look at our [Email Bridge README](https://github.com/kamax-io/matrix-appservice-email#mxisd) for an example
+of working configuration.
+
+## Configuration
+```yaml
+lookup.recursive.bridge.enabled: <boolean>
+lookup.recursive.bridge.recursiveOnly: <boolean>
+lookup.recursive.bridge.server: <URL to the bridge endpoint for all 3PID medium>
+lookup.recursive.bridge.mappings:
+  <3PID MEDIUM HERE>: <URL to dedicated bridge for that medium>
+
+```
+
+## Integration
+Implement a simplified version of the [Identity service single lookup endpoint](https://kamax.io/matrix/api/identity_service/unstable.html#get-matrix-identity-api-v1-lookup)
+with only the following parameters needed:
+- `address`
+- `medium`
+- `mxid`
+
+Or an empty object if no resolution exists or desired.

docs/features/directory-users.md

@@ -1,167 +0,0 @@
-# User Directory
-- [Description](#description)
-- [Overview](#overview)
-- [Requirements](#requirements)
-- [Configuration](#configuration)
-  - [Reverse Proxy](#reverse-proxy)
-  - [DNS Overwrite](#dns-overwrite)
-  - [Backends](#backends)
-    - [LDAP](#ldap)
-    - [SQL](#sql)
-    - [REST](#rest)
-
-## Description
-This feature allows you to search for existing and/or potential users that are already present in your Identity backend
-or that already share a room with you on the Homeserver.
-
-Without any integration, synapse:
-- Only search within the users **already** known to you
-- Only search on the Display Name and the Matrix ID
-
-With mxisd integration, you can:
-- Search on Matrix ID, Display name and 3PIDs (Email, phone numbers) of any users already in your configured backend
-- Search for users which you are not in contact with yet. Super useful for corporations who want to give Matrix access
-internally, so users can just find themselves **prior** to having any common room(s)
-- Use any attribute of your backend to extend the search!
-- Include your homeserver search results to those found by mxisd (default behaviour, no configuration required)
-
-By integrating mxisd, you get the default behaviour with all the extras, ensuring your users will always find each other.
-
-## Overview
-This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/user_directory/search` like so:
-```
-           +----------------------------------------------+
-Client --> | Reverse proxy                                                                         Step 2
-           |                                              Step 1    +-------------------------+
-           |   /_matrix/client/r0/user_directory/search ----------> |                         |  Search in   +---------+
-           |                        /\                              | mxisd - Identity server | -----------> | Backend |
-           |   /_matrix/*            \----------------------------- |                         |  all users   +---------+
-           |        |            Step 4: Send back merged results   +-------------------------+
-           +        |                                                            |
-                    |                                                          Step 3
-                    |                                                            |
-                    |    +------------+                                Search in known users
-                    \--> | Homeserver | <----------------------------------------/
-                         +------------+   /_matrix/client/r0/user_directory/search
-```
-Steps:
-1. The intercepted request is directly sent to mxisd instead of the Homeserver.
-2. Enabled backends are queried for any match on the search value sent by the client.
-3. The Homeserver, from which the request was intercepted, is queried using the same request as the client.
-Its address is resolved using the DNS Overwrite feature to reach its internal address on a non-encrypted port.
-4. Results from backends and the Homeserver are merged together and sent back to the client, believing it was the HS
-which directly answered the request.
-
-## Requirements
-- Reverse proxy setup, which you should already have in place if you use mxisd
-- Compatible backends:
-  - LDAP
-  - SQL
-  - REST
-  
-## Configuration
-### Reverse Proxy
-Apache2 configuration to put under the relevant virtual domain:
-```
-ProxyPreserveHost on
-ProxyPass /_matrix/identity/ http://mxisdInternalIpAddress:8090/_matrix/identity/
-ProxyPass /_matrix/client/r0/user_directory/ http://mxisdInternalIpAddress:8090/_matrix/client/r0/user_directory/
-ProxyPass /_matrix/ http://HomeserverInternalIpAddress:8008/_matrix/
-```
-`ProxyPreserveHost` or equivalent must be enabled to detect to which Homeserver mxisd should talk to when building
-results.
-
-### DNS Overwrite
-Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
-the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
-
-To do so, use the following configuration:
-```
-dns.overwrite.homeserver.client:
-  - name: 'example.org'
-    value: 'http://localhost:8008'
-```
-`name` must be the hostname of the URL that clients use when connecting to the Homeserver.  
-In case the hostname is the same as your Matrix domain, you can use `${matrix.domain}` to auto-populate the value using
-the `matrix.domain` configuration option and avoid duplicating it.
-
-`value` is the base intenral URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
-
-### Backends
-#### LDAP
-To ensure Directory feature works, here's how the LDAP configuration should look like:
-```
-ldap:
-  enabled: false
-  filter: '(memberOf=CN=Matrix Users,OU=Groups,DC=example,DC=org)'
-  connection:
-    host: 'ldapIpOrDomain'
-    bindDn: 'CN=Matrix Identity Server,OU=Accounts,DC=example,DC=org'
-    bindPassword: 'mxisd'
-    baseDn: 'OU=Accounts,DC=example,DC=org'
-  attribute:
-    uid:
-      type: 'uid'
-      value: 'userPrincipalName'
-    name: 'displayName'
-    threepid:
-      email:
-        - 'mailPrimaryAddress'
-        - 'mail'
-        - 'otherMailbox'
-      msisdn:
-        - 'telephoneNumber'
-        - 'mobile'
-        - 'homePhone'
-        - 'otherTelephone'
-        - 'otherMobile'
-        - 'otherHomePhone'
-  directory:
-    attribute:
-      other:
-        - 'employeeNumber'
-        - 'someOtherAttribute'
-```
-Only include the `attribute` sub-sections if you would like to set another value. Else, it is best not to include them
-to inherit the default values.
-
-If you would like to include an attribute which is not a display name or a 3PID, you can use the
-`directory.attribute.other` to list any extra attributes you want included in searches. If you do not want to include
-any extra attribute, that configuration section can be skipped. 
-
-#### SQL
-If you plan to integrate directory search directly with synapse, use the `synapseSql` provider, based on the following
-config:
-```
-synapseSql:
-  enabled: true
-  type: <database ID>
-  connection: '<connection info>'
-```
-`type` and `connection`, including any other configuration item, follow the same values as the regular [SQL backend](../backends/sql.md).
-
----
-
-For the regular SQL backend, the following configuration items are available:
-```
-sql:
-  directory:
-    enabled: true
-    query:
-      name:
-        type: 'localpart'
-        value: 'SELECT idColumn, displayNameColumn FROM table WHERE displayNameColumn LIKE ?'
-      threepid:
-        type: 'localpart'
-        value: 'SELECT idColumn, displayNameColumn FROM table WHERE threepidColumn LIKE ?'
-```
-For each query, `type` can be used to tell mxisd how to process the ID column:
-- `localpart` will append the `matrix.domain` to it
-- `mxid` will use the ID as-is. If it is not a valid Matrix ID, the search will fail.
-
-`value` is the SQL query and must return two columns:
-- The first being the User ID
-- The second being its display name
-
-#### REST
-See the [dedicated document](../backends/rest.md)

docs/features/directory.md

@@ -0,0 +1,142 @@
+# User Directory
+- [Description](#description)
+- [Overview](#overview)
+- [Requirements](#requirements)
+- [Configuration](#configuration)
+  - [Reverse Proxy](#reverse-proxy)
+    - [Apache2](#apache2)
+    - [nginx](#nginx)
+  - [DNS Overwrite](#dns-overwrite)
+- [Next steps](#next-steps)
+
+## Description
+This feature allows you to search for existing and/or potential users that are already present in your Identity backend
+or that already share a room with you on the Homeserver.
+
+Without any integration, synapse:
+- Only search within the users **already** known to you or in public rooms
+- Only search on the Display Name and the Matrix ID
+
+By enabling this feature, you can by default:
+- Search on Matrix ID, Display name and 3PIDs (Email, phone numbers) of any users already in your configured backend
+- Search for users which you are not in contact with yet. Super useful for corporations who want to give Matrix access
+internally, so users can just find themselves **prior** to having any common room(s)
+- Add extra attributes of your backend to extend the search
+- Include your homeserver search results to those found by mxisd
+
+By integrating mxisd, you get the default behaviour and a bunch of extras, ensuring your users will always find each other.
+
+## Overview
+This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/user_directory/search` like so:
+```
+           +----------------------------------------------+
+Client --> | Reverse proxy                                                                         Step 2
+           |                                              Step 1    +-------------------------+
+           |   /_matrix/client/r0/user_directory/search ----------> |                         |  Search in   +---------+
+           |                        /\                              | mxisd - Identity server | -----------> | Backend |
+           |   /_matrix/*            \----------------------------- |                         |  all users   +---------+
+           |        |            Step 4: Send back merged results   +-------------------------+
+           +        |                                                            |
+                    |                                                          Step 3
+                    |                                                            |
+                    |    +------------+                                Search in known users
+                    \--> | Homeserver | <----------------------------------------/
+                         +------------+   /_matrix/client/r0/user_directory/search
+```
+Steps:
+1. The intercepted request is directly sent to mxisd instead of the Homeserver.
+2. Identity stores are queried for any match on the search value sent by the client.
+3. The Homeserver, from which the request was intercepted, is queried using the same request as the client.
+   Its address is resolved using the DNS Overwrite feature to reach its internal address on a non-encrypted port.
+4. Results from Identity stores and the Homeserver are merged together and sent back to the client, believing it was the HS
+which directly answered the request.
+
+## Requirements
+- Reverse proxy setup, which you should already have in place if you use mxisd
+- At least one compatible [Identity store](../stores/README.md) enabled
+  
+## Configuration
+### Reverse Proxy
+#### Apache2
+The specific configuration to put under the relevant `VirtualHost`:
+```apache
+ProxyPass /_matrix/client/r0/user_directory/ http://0.0.0.0:8090/_matrix/client/r0/user_directory/
+```
+`ProxyPreserveHost` or equivalent must be enabled to detect to which Homeserver mxisd should talk to when building
+results.
+
+Your `VirtualHost` should now look like this:
+```apache
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/client/r0/user_directory/ http://localhost:8090/_matrix/client/r0/user_directory/
+    ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
+    ProxyPass /_matrix http://localhost:8008/_matrix
+</VirtualHost>
+```
+
+#### nginx
+The specific configuration to add under your `server` section is:
+```nginx
+location /_matrix/client/r0/user_directory {
+    proxy_pass http://0.0.0.0:8090/_matrix/client/r0/user_directory;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $remote_addr;
+}
+```
+
+Your `server` section should now look like this:
+```nginx
+server {
+    listen 443 ssl;
+    server_name example.org;
+    
+    ...
+    
+    location /_matrix/client/r0/user_directory {
+        proxy_pass http://localhost:8090/_matrix/client/r0/user_directory;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
+### DNS Overwrite
+Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
+the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
+
+To do so, use the following configuration:
+```yaml
+dns.overwrite.homeserver.client:
+  - name: 'example.org'
+    value: 'http://localhost:8008'
+```
+`name` must be the hostname of the URL that clients use when connecting to the Homeserver.  
+In case the hostname is the same as your Matrix domain, you can use `${matrix.domain}` to auto-populate the value using
+the `matrix.domain` configuration option and avoid duplicating it.
+
+`value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
+
+## Next steps
+### Homeserver results
+You can configure if the Homeserver should be queried at all when doing a directory search.  
+To disable Homeserver results, set the following in mxisd configuration file:
+```yaml
+directory.exclude.homeserever: true
+```

docs/features/federation.md

@@ -1,4 +1,15 @@
-# Identity service Federation
+# Federation
+Federation is the process by which domain owners can make compatible 3PIDs mapping auto-discoverable by looking for another
+Federated Identity server using the DNS domain part of the 3PID.
+
+Emails are the best candidate for this kind of resolution which are DNS domain based already.  
+On the other hand, Phone numbers cannot be resolved this way.
+
+For 3PIDs which are not compatible with the DNS system, mxisd will talk to the central Identity server of matrix.org by
+default.
+
+Outbound federation is enabled by default while inbound federation is opt-in and require a specific DNS record.
+
 ## Overview
 ```
               +-------------------+   +-------------> +----------+
@@ -17,17 +28,24 @@
                                                       | Matrix.org / Vector.im   |
                                                       +--------------------------+
 ```
-To allow other federated Identity Server to reach yours, the same algorithm used for Homeservers takes place:
-1. Check for the appropriate DNS SRV record
-2. If not found, use the base domain
 
-## Configuration
-If your Identity Server public hostname does not match your Matrix domain, configure the following DNS SRV entry 
-and replace `matrix.example.com` by your Identity server public hostname - **Make sure to end with a final dot!**
+## Inbound
+If you would like to be reachable for lookups over federation, create the following DNS SRV entry and replace
+`matrix.example.com` by your Identity server public hostname:
 ```
 _matrix-identity._tcp.example.com. 3600 IN SRV 10 0 443 matrix.example.com.
 ``` 
-This would only apply for 3PID that are DNS-based, like e-mails. For anything else, like phone numbers, no federation 
-is currently possible.  
 
 The port must be HTTPS capable which is what you get in a regular setup with a reverse proxy from 443 to TCP 8090 of mxisd.
+
+## Outbound
+If you would like to disable outbound federation and isolate your identity server from the rest of the Matrix network,
+use the following mxisd configuration options:
+```yaml
+lookup.recursive.enabled: false
+invite.resolution.recursive: false
+session.policy.validation.forLocal.toRemote.enabled: false
+session.policy.validation.forRemote.toRemote.enabled: false
+``` 
+
+There is currently no way to selectively disable federation towards specific servers, but this feature is planned.

docs/features/identity.md

@@ -1,3 +1,21 @@
-To be documented.
+# Identity
+**WARNING**: This document is incomplete and can be missleading.
 
-Implementation of the [Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html)
+Implementation of the [Unofficial Matrix Identity Service API](https://kamax.io/matrix/api/identity_service/unstable.html).
+
+## Room Invitations
+Resolution can be customized using the following configuration:
+
+`invite.resolution.recursive`  
+- Default value: `true`  
+- Description: Control if the pending invite resolution should be done recursively or not.  
+  **DANGER ZONE:** This setting has the potential to create "an isolated island", which can have unexpected side effects
+  and break invites in rooms. This will most likely not have the effect you think it does. Only change the value if you
+  understand the consequences.
+
+`invite.resolution.timer`  
+- Default value: `1`  
+- Description: How often, in minutes, mxisd should try to resolve pending invites.
+
+## 3PID addition to user profile
+See the [3PID session documents](../threepids/session)

docs/features/profile.md

@@ -0,0 +1,9 @@
+# Profile enhancement
+**WARNING**: Alpha feature not officially supported. Do not use.
+
+## Configuration
+### Reverse proxy
+#### Apache
+```apache
+ProxyPassMatch "^/_matrix/client/r0/profile/([^/]+)$" "http://127.0.0.1:8090/_matrix/client/r0/profile/$1"
+```

docs/getting-started.md

@@ -0,0 +1,135 @@
+# Getting started
+1. [Preparation](#preparation)
+2. [Install](#install)
+3. [Configure](#configure)
+4. [Integrate](#integrate)
+5. [Validate](#validate)
+6. [Next steps](#next-steps)
+
+Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
+talk to the central Matrix.org Identity server.  
+This will be a good ground work for further integration with features and your existing Identity stores.
+
+## Preparation
+You will need:
+- Homeserver
+- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
+
+As synapse requires an HTTPS connection when talking to an Identity service, **a reverse proxy is required** as mxisd does
+not support HTTPS listener at this time.
+
+For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.
+
+Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-io/mxisd/wiki/Gotchas#nating) if you use the same
+hostname.
+
+The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
+If you would like a high-level view of the infrastructure and how each feature is integrated, see the
+[dedicated document](architecture.md)
+
+## Install
+Install via:
+- [Debian package](install/debian.md)
+- [Docker image](install/docker.md)
+- [Sources](build.md)
+
+See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
+
+## Configure
+**NOTE**: please view the install instruction for your platform, as this step might be optional or already handled for you.
+
+Create/edit a minimal configuration (see installer doc for the location):
+```yaml
+matrix.domain: 'example.org'
+key.path: '/path/to/signing.key.file'
+storage.provider.sqlite.database: '/path/to/mxisd.db'
+```  
+- `matrix.domain` should be set to your Homeserver domain (`server_name` in synapse configuration)
+- `key.path` will store the signing keys, which must be kept safe! If the file does not exist, keys will be generated for you.
+- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
+
+If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
+Complete configuration guide is available [here](configure.md).
+
+## Integrate
+For an overview of a typical mxisd infrastructure, see the [dedicated document](architecture.md)
+### Reverse proxy
+#### Apache2
+In the `VirtualHost` section handling the domain with SSL, add the following and replace `0.0.0.0` by the internal
+hostname/IP pointing to mxisd.  
+**This line MUST be present before the one for the homeserver!**
+```apache
+ProxyPass /_matrix/identity http://0.0.0.0:8090/_matrix/identity
+```
+
+Typical configuration would look like:
+```apache
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/identity http://localhost:8090/_matrix/identity
+    ProxyPass /_matrix http://localhost:8008/_matrix
+</VirtualHost>
+```
+
+#### nginx
+In the `server` section handling the domain with SSL, add the following and replace `0.0.0.0` with the internal
+hostname/IP pointing to mxisd.
+**This line MUST be present before the one for the homeserver!**
+```nginx
+location /_matrix/identity {
+    proxy_pass http://0.0.0.0:8090/_matrix/identity;
+}
+```
+
+Typical configuration would look like:
+```nginx
+server {
+    listen 443 ssl;
+    server_name example.org;
+    
+    ...
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
+### Synapse
+Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
+In a typical configuration, you would end up with something similar to:
+```yaml
+trusted_third_party_id_servers:
+    - example.org
+```
+It is recommended to remove `matrix.org` and `vector.im` (or any other default entry) from your configuration so only
+your own Identity server is authoritative for your HS.
+
+## Validate
+Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
+the relevant hostname which you configured in your reverse proxy.  
+Invite `mxisd-federation-test@kamax.io` to a room, which should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.
+At this point, the test user will join the room, send a congratulation message and leave.  
+**NOTE:** You might not see a suggestion for the e-mail address, which is normal. Still proceed with the invite.
+  
+If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
+If it did not work, [get in touch](../README.md#support) and we'll do our best to get you started.
+
+## Next steps
+Once your mxisd server is up and running, there are several ways you can enhance and integrate further with your
+infrastructure:
+
+- [Enable extra features](features/)
+- [Use your own Identity stores](stores/README.md)

docs/install/archlinux.md

@@ -0,0 +1,3 @@
+# Arch Linux package
+An Arch Linux package in the AUR repos is maintained by [r3pek](https://matrix.to/#/@r3pek:r3pek.org), a community member.  
+See https://aur.archlinux.org/packages/mxisd/

docs/install/debian.md

@@ -1,28 +1,28 @@
 # Debian package
 ## Install
-1. Donwload the [latest release](https://github.com/kamax-io/mxisd/releases/latest)
+1. Download the [latest release](https://github.com/kamax-io/mxisd/releases/latest)
 2. Run:
-```
+```bash
 dpkg -i /path/to/downloaded/mxisd.deb
 ```
 ## Files
 | Location                            | Purpose                                      |
-|-----------------------------------|----------------------------------------------|
-| /etc/mxisd                        | Configuration directory                      |
-| /etc/mxisd/mxisd.yaml             | Main configuration file                      |
-| /etc/mxisd/signing.key            | Default location for mxisd signing keys      |
-| /etc/systemd/system/mxisd.service | Systemd configuration file for mxisd service |
-| /usr/lib/mxisd                    | Binairies                                    |
-| /var/lib/mxisd                    | Data                                         |
+|-------------------------------------|----------------------------------------------|
+| `/etc/mxisd`                        | Configuration directory                      |
+| `/etc/mxisd/mxisd.yaml`             | Main configuration file                      |
+| `/etc/systemd/system/mxisd.service` | Systemd configuration file for mxisd service |
+| `/usr/lib/mxisd`                    | Binaries                                     |
+| `/var/lib/mxisd`                    | Data                                         |
+| `/var/lib/mxisd/signing.key`        | Default location for mxisd signing keys      |
 
 ## Control
 Start mxisd using:
-```
+```bash
 sudo systemctl start mxisd
 ```
 
 Stop mxisd using:
-```
+```bash
 sudo systemctl stop mxisd
 ```
 
@@ -35,5 +35,5 @@ tail -n 99 -f /var/log/syslog | grep mxisd
 ```
 - use Systemd's journal:
 ```
-journalctl -f n 99 -u mxisd
+journalctl -f -n 99 -u mxisd
 ```

docs/install/docker.md

@@ -1,14 +1,22 @@
 # Docker
 ## Fetch
 Pull the latest stable image:
-```
+```bash
 docker pull kamax/mxisd
 ```
 
+## Configure
+On first run, simply using `MATRIX_DOMAIN` as an environment variable will create a default config for you.  
+You can also provide a configuration file named `mxisd.yaml` in the volume mapped to `/etc/mxisd` before starting your
+container.
+
 ## Run
-Run it (adapt volume paths to your host):
-```
-docker run --rm -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
+Use the following command after adapting to your needs:
+- The `MATRIX_DOMAIN` environment variable to yours
+- The volumes host paths
+
+```bash
+docker run --rm -e MATRIX_DOMAIN=example.org -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
 ```
 
 For more info, including the list of possible tags, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)

docs/install/source.md

@@ -0,0 +1,38 @@
+# Install from sources
+## Instructions
+Follow the [build instructions](../build.md) then:
+
+1. Prepare files and directories:
+```bash
+# Create a dedicated user
+useradd -r mxisd
+
+# Create bin directory
+mkdir /opt/mxisd
+
+# Create config directory and set ownership
+mkdir -p /etc/opt/mxisd
+chown -R mxisd /etc/opt/mxisd
+
+# Create data directory and set ownership
+mkdir -p /var/opt/mxisd
+chown -R mxisd /var/opt/mxisd
+
+# Copy <repo root>/build/libs/mxisd.jar to bin directory
+cp ./build/libs/mxisd.jar /opt/mxisd/
+chown mxisd /opt/mxisd/mxisd.jar
+chmod a+x /opt/mxisd/mxisd.jar
+
+# Create symlink for easy exec
+ln -s /opt/mxisd/mxisd.jar /usr/bin/mxisd
+```
+2. Copy the sample config file `./application.example.yaml` to `/etc/opt/mxisd/mxisd.yaml`, edit to your needs
+3. Copy `src/systemd/mxisd.service` to `/etc/systemd/system/` and edit if needed
+4. Enable service for auto-startup
+```bash
+systemctl enable mxisd
+```
+5. Start mxisd
+```bash
+systemctl start mxisd
+```

docs/stores/README.md

@@ -0,0 +1,7 @@
+# Identity Stores
+- [Synapse](synapse.md)
+- [LDAP-based](ldap.md)
+- [SQL Databases](sql.md)
+- [Website / Web service / Web app](rest.md)
+- [Google Firebase](firebase.md)
+- [Wordpress](wordpress.md)

docs/stores/firebase.md

@@ -0,0 +1,52 @@
+# Google Firebase Identity store
+https://firebase.google.com/
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | No         |
+| Identity       | Yes        |
+
+## Requirements
+This backend requires a suitable Matrix client capable of performing Firebase authentication and passing the following
+information:
+- Firebase User ID as Matrix username
+- Firebase token as Matrix password
+
+If your client is Riot, you will need a custom version.
+
+## Configuration
+```yaml
+firebase.enabled: <boolean>
+```
+Enable/disable this identity store.
+
+Example:
+```yaml
+firebase.enabled: <boolean>
+```
+
+---
+
+```yaml
+firebase.credentials: <string>
+```
+Path to the credentials file provided by Google Firebase to use with an external app.
+
+Example:
+```yaml
+firebase.credentials: '/path/to/firebase/credentials.json'
+```
+
+---
+
+```yaml
+firebase.database: <string>
+```
+URL to your Firebase database.
+
+Example:
+```yaml
+firebase.database: 'https://my-project.firebaseio.com/'
+```

docs/stores/ldap.md

@@ -0,0 +1,120 @@
+# LDAP Identity store
+## Supported products:
+- Samba
+- Active Directory
+- OpenLDAP
+- NetIQ eDirectory
+
+For NetIQ, replace all the `ldap` prefix in the configuration by `netiq`.
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | Yes        |
+| Identity       | Yes        |
+
+## Getting started
+### Base
+To use your LDAP backend, add the bare minimum configuration in mxisd config file:
+```yaml
+ldap.enabled: true
+ldap.connection.host: 'ldapHostnameOrIp'
+ldap.connection.port: 389
+ldap.connection.bindDn: 'CN=My Mxisd User,OU=Users,DC=example,DC=org'
+ldap.connection.bindPassword: 'TheUserPassword'
+ldap.connection.baseDn: 'OU=Users,DC=example,DC=org'
+```
+These are standard LDAP connection configuration. mxisd will try to connect on port default port 389 without encryption.
+
+### TLS/SSL connection
+If you would like to use a TLS/SSL connection, use the following configuration options (STARTLS not supported):
+```yaml
+ldap.connection.tls: true
+ldap.connection.port: 12345
+```
+
+### Filter results
+You can also set a default global filter on any LDAP queries:
+```yaml
+ldap.filter: '(memberOf=CN=My Matrix Users,OU=Groups,DC=example,DC=org)'
+```
+This example would only return users part of the group called `My Matrix Users`.
+This can be overwritten or append in each specific flow describe below.
+
+For supported syntax, see the [LDAP library documentation](http://directory.apache.org/api/user-guide/2.3-searching.html#filter).
+
+### Attribute mapping
+LDAP features are based on mapping LDAP attributes to Matrix concepts, like a Matrix ID, its localpart, the user display
+name, their email(s) and/or phone number(s).
+     
+Default attributes are well suited for Active Directory/Samba. In case you are using a native LDAP backend, you will
+most certainly configure those mappings.
+
+#### User ID
+`ldap.attribute.uid.type`: How to process the User ID (UID) attribute:
+- `uid` will consider the value as the [Localpart](https://matrix.org/docs/spec/intro.html#user-identifiers)
+- `mxid` will consider the value as a complete [Matrix ID](https://matrix.org/docs/spec/intro.html#user-identifiers)
+
+`ldap.attribute.uid.value`: Attribute to use to set the User ID value.
+
+The following example would set the `sAMAccountName` attribute as a Matrix User ID localpart:
+```yaml
+ldap.attribute.uid.type: 'uid'
+ldap.attribute.uid.value: 'sAMAccountName'
+```
+
+#### Display name
+Use `ldap.attribute.name`.
+
+The following example would set the display name to the value of the `cn` attribute:
+```yaml
+ldap.attribute.name: 'cn'
+```
+
+#### 3PIDs
+You can also change the attribute lists for 3PID, like email or phone numbers.
+
+The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67)
+for emails and phone number:
+```yaml
+ldap.attribute.threepid.email:
+  - 'mail'
+  - 'otherMailAttribute'
+
+ldap.attribute.threepid.msisdn:
+  - 'phone'
+  - 'otherPhoneAttribute'
+```
+
+## Features
+### Identity
+Identity features (related to 3PID invites or searches) are enabled and configured using default values and no specific
+configuration item is needed to get started.
+
+#### Configuration
+- `ldap.identity.filter`: Specific user filter applied during identity search. Global filter is used if blank/not set.
+- `ldap.identity.medium`: Namespace to overwrite generated queries from the list of attributes for each 3PID medium.
+
+### Authentication
+No further configuration is needed to use the Authentication feature with LDAP once globally enabled and configured.
+
+Profile auto-fill is enabled by default. It will use the `ldap.attribute.name` and `ldap.attribute.threepid` configuration
+options to get a lit of attributes to be used to build the user profile to pass on to synapse during authentication.
+
+#### Configuration
+- `ldap.auth.filter`: Specific user filter applied during identity search. Global filter is used if blank/not set.
+
+### Directory
+No further configuration is needed to use the Directory feature with LDAP once globally enabled and configured.
+
+#### Configuration
+To set a specific filter applied during directory search, use `ldap.directory.filter`
+
+If you would like to use extra attributes in search that are not 3PIDs, like nicknames, group names, employee number:
+```yaml
+ldap.directory.attribute.other:
+  - 'myNicknameAttribute'
+  - 'memberOf'
+  - 'employeeNumberAttribute'
+```

docs/stores/rest.md

@@ -1,27 +1,28 @@
-# REST backend
+# REST Identity store
 The REST backend allows you to query identity data in existing webapps, like:
 - Forums (phpBB, Discourse, etc.)
 - Custom Identity stores (Keycloak, ...)
 - CRMs (Wordpress, ...)
 - self-hosted clouds (Nextcloud, ownCloud, ...)
 
-It supports the following mxisd flows:
-- [Authentication](#authentication)
-- [Directory](#directory)
-- [Identity](#identity)
-
 To integrate this backend with your webapp, you will need to implement three specific REST endpoints detailed below.
 
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | Yes        |
+| Identity       | Yes        |
 
 ## Configuration
 | Key                              | Default                                        | Description                                          |
----------------------------------|----------------------------------------------|------------------------------------------------------|
-| rest.enabled                   | false                                        | Globally enable/disable the REST backend             |
-| rest.host                      | *empty*                                      | Default base URL to use for the different endpoints. |
-| rest.endpoints.auth            | /_mxisd/backend/api/v1/auth/login            | Validate credentials and get user profile            |
-| rest.endpoints.directory       | /_mxisd/backend/api/v1/directory/user/search | Search for users by arbitrary input                  |
-| rest.endpoints.identity.single | /_mxisd/backend/api/v1/identity/single       | Endpoint to query a single 3PID                      |
-| rest.endpoints.identity.bulk   | /_mxisd/backend/api/v1/identity/bulk         | Endpoint to query a list of 3PID                     |
+|----------------------------------|------------------------------------------------|------------------------------------------------------|
+| `rest.enabled`                   | `false`                                        | Globally enable/disable the REST backend             |
+| `rest.host`                      | *None*                                        | Default base URL to use for the different endpoints. |
+| `rest.endpoints.auth`            | `/_mxisd/backend/api/v1/auth/login`            | Validate credentials and get user profile            |
+| `rest.endpoints.directory`       | `/_mxisd/backend/api/v1/directory/user/search` | Search for users by arbitrary input                  |
+| `rest.endpoints.identity.single` | `/_mxisd/backend/api/v1/identity/single`       | Endpoint to query a single 3PID                      |
+| `rest.endpoints.identity.bulk`   | `/_mxisd/backend/api/v1/identity/bulk`         | Endpoint to query a list of 3PID                     |
 
 Endpoint values can handle two formats:
 - URL Path starting with `/` that gets happened to the `rest.host`
@@ -35,7 +36,7 @@ HTTP method: `POST`
 Content-type: JSON UTF-8
   
 #### Request Body
-```
+```json
 {
   "auth": {
     "mxid": "@john.doe:example.org",
@@ -48,7 +49,7 @@ Content-type: JSON UTF-8
 
 #### Response Body
 If the authentication fails:
-```
+```json
 {
   "auth": {
     "success": false
@@ -59,7 +60,7 @@ If the authentication fails:
 If the authentication succeed:
 - `auth.id` supported values: `localpart`, `mxid`
 - `auth.profile` and any sub-member are all optional
-```
+```json
 {
   "auth": {
     "success": true,
@@ -89,7 +90,7 @@ HTTP method: `POST`
 Content-type: JSON UTF-8
 
 #### Request Body
-```
+```json
 {
   "by": "<search type>",
   "search_term": "doe"
@@ -101,7 +102,7 @@ Content-type: JSON UTF-8
 
 #### Response Body:
 If users found:
-```
+```json
 {
   "limited": false,
   "results": [
@@ -118,7 +119,7 @@ If users found:
 ```
 
 If no user found:
-```
+```json
 {
   "limited": false,
   "results": []
@@ -131,7 +132,7 @@ HTTP method: `POST`
 Content-type: JSON UTF-8  
   
 #### Request Body
-```
+```json
 {
   "lookup": {
     "medium": "email",
@@ -143,7 +144,7 @@ Content-type: JSON UTF-8
 #### Response Body
 If a match was found:
 - `lookup.id.type` supported values: `localpart`, `mxid`
-```
+```json
 {
   "lookup": {
     "medium": "email",
@@ -157,7 +158,7 @@ If a match was found:
 ```
 
 If no match was found:
-```
+```json
 {}
 ```
 
@@ -166,7 +167,7 @@ HTTP method: `POST`
 Content-type: JSON UTF-8  
   
 #### Request Body
-```
+```json
 {
   "lookup": [
     {
@@ -184,7 +185,7 @@ Content-type: JSON UTF-8
 #### Response Body
 For all entries where a match was found:
 - `lookup[].id.type` supported values: `localpart`, `mxid`
-```
+```json
 {
   "lookup": [
     {
@@ -208,7 +209,7 @@ For all entries where a match was found:
 ```
 
 If no match was found:
-```
+```json
 {
   "lookup": []
 }

docs/stores/sql.md

@@ -0,0 +1,98 @@
+# SQL Identity store
+## Supported Databases
+- PostgreSQL
+- MariaDB
+- MySQL
+- SQLite
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | No         |
+| Directory      | Yes        |
+| Identity       | Yes        |
+
+Due to the implementation complexity of supporting arbitrary hashing/encoding mechanisms or auth flow, Authentication
+will be out of scope of SQL Identity stores and should be done via one of the other identity stores, typically
+the [REST Identity store](rest.md).
+
+## Configuration
+### Basic
+```yaml
+sql.enabled: <boolean>
+```
+Enable/disable the identity store
+
+---
+
+```yaml
+sql.type: <string>
+```
+Set the SQL backend to use:
+- `sqlite`
+- `postgresql`
+- `mariadb`
+- `mysql`
+
+### Connection
+#### SQLite
+```yaml
+sql.connection: <string>
+```
+Set the value to the absolute path to the Synapse SQLite DB file.
+Example: `/path/to/sqlite/file.db`
+
+#### Others
+```yaml
+sql.connection: //<HOST[:PORT]/DB?username=USER&password=PASS
+```
+Set the connection info for the database by replacing the following values:
+- `HOST`: Hostname of the SQL server
+- `PORT`: Optional port value, if not default
+- `DB`: Database name
+- `USER`: Username for the connection
+- `PASS`: Password for the connection
+
+This follow the JDBC URI syntax. See [official website](https://docs.oracle.com/javase/tutorial/jdbc/basics/connecting.html#db_connection_url).
+
+### Directory
+```yaml
+sql.directory.enabled: false
+```
+
+
+---
+
+```yaml
+sql.directory.query:
+  name:
+    type: <string>
+    value: <string>
+  threepid:
+    type: <string>
+    value: <string>
+```
+For each query, `type` can be used to tell mxisd how to process the ID column:
+- `localpart` will append the `matrix.domain` to it
+- `mxid` will use the ID as-is. If it is not a valid Matrix ID, the search will fail.
+
+`value` is the SQL query and must return two columns:
+- The first being the User ID
+- The second being its display name
+
+Example:
+```yaml
+sql.directory.query:
+  name:
+    type: 'localpart'
+    value: 'SELECT idColumn, displayNameColumn FROM table WHERE displayNameColumn LIKE ?'
+  threepid:
+    type: 'localpart'
+    value: 'SELECT idColumn, displayNameColumn FROM table WHERE threepidColumn LIKE ?'
+```
+
+### Identity
+```yaml
+sql.identity.type: <string>
+sql.identity.query: <string>
+```

docs/stores/synapse.md

@@ -0,0 +1,48 @@
+# Synapse Identity Store
+Synapse's Database itself can be used as an Identity store.
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | No         |
+| Directory      | Yes        |
+| Identity       | Yes        |
+
+Authentication is done by Synapse itself.
+
+## Configuration
+### Basic
+```yaml
+synapseSql.enabled: <boolean>
+```
+Enable/disable the identity store
+
+---
+
+```yaml
+synapseSql.type: <string>
+```
+Set the SQL backend to use which is configured in synapse:
+- `sqlite`
+- `postgresql`
+
+### SQLite
+```yaml
+synapseSql.connection: <string>
+```
+Set the value to the absolute path to the Synapse SQLite DB file.
+Example: `/path/to/synapse/sqliteFile.db`
+
+### PostgreSQL
+```yaml
+synapseSql.connection: //<HOST[:PORT]/DB?username=USER&password=PASS
+```
+Set the connection info for the database by replacing the following values:
+- `HOST`: Hostname of the SQL server
+- `PORT`: Optional port value, if not default
+- `DB`: Database name
+- `USER`: Username for the connection
+- `PASS`: Password for the connection
+
+### Query customization
+See the [SQL Identity store](sql.md)

docs/stores/wordpress.md

@@ -0,0 +1,57 @@
+# Wordpress Identity store
+This Identity store allows you to use user accounts registered on your Wordpress setup.  
+Two types of connections are required for full support:
+- [REST API](https://developer.wordpress.org/rest-api/) with JWT authentication
+- Direct SQL access
+
+## Features
+|      Name      | Supported? |
+|----------------|------------|
+| Authentication | Yes        |
+| Directory      | Yes        |
+| Identity       | Yes        |
+
+## Requirements
+- [Wordpress](https://wordpress.org/download/) >= 4.4
+- Permalink structure set to `Post Name`
+- [JWT Auth plugin for REST API](https://wordpress.org/plugins/jwt-authentication-for-wp-rest-api/)
+- SQL Credentials to the Wordpress Database
+
+## Configuration
+### Wordpress
+#### JWT Auth
+Set a JWT secret into `wp-config.php` like so:
+```php
+define('JWT_AUTH_SECRET_KEY', 'your-top-secret-key');
+```
+`your-top-secret-key` should be set to a randomly generated value which is kept secret.
+
+#### Rewrite of `index.php`
+Wordpress is normally configured with rewrite of `index.php` so it does not appear in URLs.  
+If this is not the case for your installation, the mxisd URL will need to be appended with `/index.php`
+
+### mxisd
+Enable in the configuration:
+```yaml
+wordpress.enabled: true
+```
+Configure the URL to your Wordpress installation - see above about added `/index.php`:
+```yaml
+wordpress.rest.base: 'http://localhost:8080'
+```
+Configure the SQL connection to your Wordpress database:
+```yaml
+wordpress.sql.connection: '//127.0.0.1/wordpress?user=root&password=example'
+```
+
+---
+
+By default, MySQL database is expected. If you use another database, use:
+```yaml
+wordpress.sql.type: <string>
+```
+With possible values:
+- `mysql`
+- `mariadb`
+- `postgresql`
+- `sqlite`

docs/threepids/medium/email/smtp-connector.md

@@ -1,11 +1,11 @@
 # Email notifications - SMTP connector
+Enabled by default.
+
 Connector ID: `smtp`
 
-Example configuration:
-```
-threepid:
-  medium:
-    email:
+## Configuration
+```yaml
+threepid.medium.email:
   identity:
     from: 'identityServerEmail@example.org'
     name: 'My Identity Server'

docs/threepids/medium/msisdn/twilio-connector.md

@@ -1,15 +1,11 @@
 # SMS notifications - Twilio connector
+Enabled by default.
+
 Connector ID: `twilio`
 
-Example configuration:
-```
-threepid:
-  medium:
-    msisdn:
-      connectors:
-        twilio:
-          accountSid: 'myAccountSid'
-          authToken: 'myAuthToken'
-          number: '+123456789'
-
+## Configuration
+```yaml
+threepid.medium.msisdn.connectors.twilio.accountSid: 'myAccountSid'
+threepid.medium.msisdn.connectors.twilio.authToken: 'myAuthToken'
+threepid.medium.msisdn.connectors.twilio.number: '+123456789'
 ```

docs/threepids/notification/basic-handler.md

@@ -0,0 +1,37 @@
+# Basic Notification handler
+Basic notification handler which uses two components:
+- Content generator, to produce the notifications
+- Connectors to send the notification content
+
+This handler can be used with the 3PID types:
+- `email`
+- `msisdn` (Phone numbers)
+
+## Generators
+- [Template](template-generator.md)
+## Connectors
+- Email
+  - [SMTP](../medium/email/smtp-connector.md)
+- SMS
+  - [Twilio](../medium/msisdn/twilio-connector.md)
+
+## Configuration
+Enabled by default or with:
+```yaml
+notification.handler.email: 'raw'
+```
+
+**WARNING:** Will be consolidated soon, prone to breaking changes.  
+Structure and default values:
+```yaml
+threepid.medium:
+  email:
+    identity:
+      from: ''
+      name: ''
+    connector: 'smtp'
+    generator: 'template'
+  msisdn:
+    connector: 'twilio'
+    generator: 'template'
+```

docs/threepids/notification/sendgrid-handler.md

@@ -2,8 +2,6 @@
 To be completed. See [raw possible configuration items](https://github.com/kamax-io/mxisd/blob/master/src/main/resources/application.yaml#L172).
 
 Enabled with:
-```
-notification:
-  handler:
-    email: 'sendgrid'
+```yaml
+notification.handler.email: 'sendgrid'
 ```

docs/threepids/notification/template-generator.md

@@ -10,10 +10,8 @@ placeholders and also have their own individual set of placeholders.
 
 ## Configuration
 To configure paths to the various templates:
-```
-threepid:
-  medium:
-    <YOUR 3PID MEDIUM HERE>:
+```yaml
+threepid.medium.<YOUR 3PID MEDIUM HERE>:
   generators:
     template:
       invite: '/path/to/invite-template.eml'

docs/threepids/notifications/basic-handler.md

@@ -1,66 +0,0 @@
-# Basic Notification handler
-Basic notification handler which uses two components:
-- Content generator, to produce the notifications
-- Connectors to send the notification content
-
-This handler can be used with the 3PID types:
-- `email`
-- `msisdn` (Phone numbers)
-
-## Generators
-- [Template](template-generator.md)
-## Connectors
-- Email
-  - [SMTP](../medium/email/smtp-connector.md)
-- SMS
-  - [Twilio](../medium/msisdn/twilio-connector.md)
-
-## Configuration
-Enabled by default or with:
-```
-notification:
-  handler:
-    email: 'raw'
-```
-
-**WARNING:** Will be consolidated soon, prone to breaking changes.  
-Structure and default values:
-```
-threepid:
-  medium:
-    email:
-      identity:
-        from: ''
-        name: ''
-      connector: 'smtp'
-      generator: 'template'
-      connectors:
-        smtp:
-          host: ''
-          port: 587
-          tls: 1
-          login: ''
-          password: ''
-      generators:
-        template:
-          invite: 'classpath:threepids/email/invite-template.eml'
-          session:
-            validation:
-              local: 'classpath:threepids/email/validate-local-template.eml'
-              remote: 'classpath:threepids/email/validate-remote-template.eml'
-    msisdn:
-      connector: 'twilio'
-      generator: 'template'
-      connectors:
-        twilio:
-          accountSid: ''
-          authToken: ''
-          number: ''
-      generators:
-        template:
-          invite: 'classpath:threepids/sms/invite-template.txt'
-          session:
-            validation:
-              local: 'classpath:threepids/sms/validate-local-template.txt'
-              remote: 'classpath:threepids/sms/validate-remote-template.txt'
-```

docs/threepids/session/session-views.md

@@ -1,26 +1,30 @@
-# Web pages for the 3PID session processes
+# Web pages for the 3PID sessions
 You can customize the various pages used during a 3PID validation using [Thymeleaf templates](http://www.thymeleaf.org/).
 
 ## Configuration
-```
-view:
-  session:
-    local:
+Pseudo-configuration to illustrate the structure:
+```yaml
+# CONFIGURATION EXAMPLE
+# DO NOT COPY/PASTE THIS IN YOUR CONFIGURATION
+view.session.local:
   onTokenSubmit:
     success: '/path/to/session/local/tokenSubmitSuccess-page.html'
     failure: '/path/to/session/local/tokenSubmitFailure-page.html'
-    localRemote:
+view.session.localRemote:
   onTokenSubmit:
     success: '/path/to/session/localRemote/tokenSubmitSuccess-page.html'
     failure: '/path/to/session/local/tokenSubmitFailure-page.html'
-    remote:
+view.session.remote:
   onRequest:
     success: '/path/to/session/remote/requestSuccess-page.html'
     failure: '/path/to/session/remote/requestFailure-page.html'
   onCheck:
     success: '/path/to/session/remote/checkSuccess-page.html'
     failure: '/path/to/session/remote/checkFailure-page.html'
+# CONFIGURATION EXAMPLE
+# DO NOT COPY/PASTE THIS IN YOUR CONFIGURATION
 ```
+
 3PID session are divided into three config sections:
 - `local` for local-only 3PID sessions
 - `localRemote` for local 3PID sessions that can also be turned into remote sessions, if the user so desires

docs/threepids/session/session.md

@@ -6,7 +6,7 @@
   - [Session scope](#session-scope)
 - [Notifications](#notifications)
   - [Email](#email)
-  - [Phone numbers](#msisdn-phone-numbers)
+  - [Phone numbers](#msisdn-(phone-numbers))
 - [Usage](#usage)
   - [Configuration](#configuration)
   - [Web views](#web-views)
@@ -47,7 +47,7 @@ To ensure lookup works consistency within the current Matrix network, the centra
 used to store *remote* sessions and binds.
 
 On the flip side, at the time of writing, the Matrix specification and the central Matrix.org servers do not allow to
-remote a 3PID bind. This means that once a 3PID is published (email, phone number, etc.), it cannot be easily remove
+remote a 3PID bind. This means that once a 3PID is published (email, phone number, etc.), it cannot be easily removed
 and would require contacting the Matrix.org administrators for each bind individually.  
 This poses a privacy, control and security concern, especially for groups/corporations that want to keep a tight control
 on where such identifiers can be made publicly visible.
@@ -80,9 +80,8 @@ Sessions can be scoped as:
 
 **IMPORTANT NOTE:** mxisd does not store bindings directly. While a user can see its email, phone number or any other
 3PID in its settings/profile, it does **NOT** mean it is published anywhere and can be used to invite/search the user.
-Identity backends (LDAP, REST, SQL) are the ones holding such data.  
-If you still want added arbitrary 3PIDs to be discoverable on your local server, you will need to link mxisd to your
-synapse DB to make it an Identity backend.
+Identity stores are the ones holding such data.  
+If you still want added arbitrary 3PIDs to be discoverable on a synapse Homeserver, use the corresponding [Identity store](../../stores/synapse.md).
 
 See the [Scenarios](#scenarios) for more info on how and why.
 
@@ -99,17 +98,17 @@ Built-in generators and connectors for supported 3PID types:
 
 ### Email
 Generators:
-- [Template](../threepids/notifications/template-generator.md)
+- [Template](../notification/template-generator.md)
 
 Connectors:
-- [SMTP](../threepids/medium/email/smtp-connector.md)
+- [SMTP](../medium/email/smtp-connector.md)
 
 #### MSISDN (Phone numbers)
 Generators:
-- [Template](../threepids/notifications/template-generator.md)
+- [Template](../notification/template-generator.md)
 
 Connectors:
- - [Twilio](../threepids/medium/msisdn/twilio-connector.md) with SMS
+ - [Twilio](../medium/msisdn/twilio-connector.md) with SMS
 
 ## Usage
 ### Configuration
@@ -117,76 +116,24 @@ The following example of configuration (incomplete extract) shows which items ar
 
 **IMPORTANT:** Most configuration items shown have default values and should not be included in your own configuration
 file unless you want to specifically overwrite them.
-Please refer to the full example config file to see which keys are mandatory and to be included in your configuration.
-```
-matrix:
-  identity:
-    servers:
-      configExample: # Not to be included in config! Already present in default config!
-        - 'https://example.org'
-
-
-threepid:
-  medium:
-    email:
-      connector: 'example1' # Not to be included in config! Already present in default config! 
-      generator: 'example2' # Not to be included in config! Already present in default config!
-      connectors:
-        example1:
-      generators:
-        example1:
-          key: "value"
-        example2:
-          key: "value"
-
-session:
-  policy:
-    validation:
-      enabled: true
-      forLocal:
+```yaml
+# DO NOT COPY/PASTE THIS IN YOUR CONFIGURATION
+session.policy.validation.enabled: true
+session.policy.validation.forLocal:
   enabled: true
   toLocal: true
   toRemote:
     enabled: true
     server: 'configExample'  # Not to be included in config! Already present in default config!
-      forRemote:
+session.policy.validation.forRemote:
   enabled: true
-        toLocal: false
+  toLocal: true
   toRemote:
     enabled: true
     server: 'configExample'  # Not to be included in config! Already present in default config!
+# DO NOT COPY/PASTE THIS IN YOUR CONFIGURATION
 ```
 
-`matrix.identity.servers` is the namespace to configure arbitrary list of Identity servers with a label as parent key.  
-In the above example, the list with label `configExample` contains a single server entry pointing to `https://example.org`.  
-
-**NOTE:** The server list is set to `root` by default and should typically NOT be included in your config.  
-
-Identity server entry can be of two format:
-- URL, bypassing any kind of domain and port discovery
-- Domain name as `string`, allowing federated discovery to take place.
-
-The label can be used in other places of the configuration, allowing you to only declare Identity servers once.
-
----
-
-`threepid.medium.<3PID>` is the namespace to configure 3PID specific items, not directly tied to any other component of
-mxisd.  
-In the above example, only `email` is defined as 3PID type.
-
-Each 3PID namespace comes with 4 configuration key allowing you to configure generators and connectors for notifications:
-- `connectors` is a configuration namespace to be used for any connector configuration. Child keys represent the unique
-ID for each connector.
-- `generators` is a configuration namespace to be used for any generator configuration. Child keys represent the unique
-ID for each generator.
-- `connector` is given the ID of the connector to be used at runtime.
-- `generator` is given the ID of the generator to be used at runtime.
-
-In the above example, emails notifications are generated by the `example2` module and sent with the `example1` module.  
-By default, `template` is used as generator and `smtp` as connector.
-
----
-
 `session.policy.validation` is the core configuration to control what users configured to use your Identity server
 are allowed to do in terms of 3PID sessions.
 
@@ -203,16 +150,16 @@ If both `toLocal` and `toRemote` are enabled, the user will be offered to initia
 locally validated.
 
 ### Web views
-Once a user click on a validation link, it is taken to the Identity Server validation page where the token is submited.  
+Once a user click on a validation link, it is taken to the Identity Server validation page where the token is submitted.  
 If the session or token is invalid, an error page is displayed.  
 Workflow pages are also available for the remote 3PID session process.
 
-See [the dedicated document](3pid-views.md)
+See [the dedicated document](session-views.md)
 on how to configure/customize/brand those pages to your liking.
 
 ### Scenarios
 It is important to keep in mind that mxisd does not create bindings, irrelevant if a user added a 3PID to their profile.  
-Instead, when queried for bindings, mxisd will query Identity backends which are responsible to store this kind of information.
+Instead, when queried for bindings, mxisd will query Identity stores which are responsible to store this kind of information.
 
 This has the side effect that any 3PID added to a user profile which is NOT within a configured and enabled Identity backend
 will simply not be usable for search or invites, **even on the same Homeserver!**  
@@ -220,8 +167,7 @@ mxisd does not store binds on purpose, as one of its primary goal is to ensure m
 and the rest of the Matrix ecosystem is preserved.
 
 Nonetheless, because mxisd also aims at offering support for tight control over identity data, it is possible to have
-such 3PID bindings available for search and invite queries on the local Homeserver by using the `SQL` backend and
-configuring it to use the synapse database. Support for `SQLite` and `PostgreSQL` is available.
+such 3PID bindings available for search and invite queries on synapse with the corresponding [Identity store](../../stores/synapse.md).
 
 See the [Local sessions only](#local-sessions-only) use case for more information on how to configure.
 
@@ -229,7 +175,7 @@ See the [Local sessions only](#local-sessions-only) use case for more informatio
 By default, mxisd allows the following:
 
 |                 | Local Session     | Remote Session |
-|----------------|-------|--------|
+|-----------------|-------------------|----------------|
 | **Local 3PID**  | Yes               | Yes, offered   |
 | **Remote 3PID** | No, Remote forced | Yes            |
 
@@ -243,8 +189,8 @@ Other e-mail addresses and phone number will be redirected to remote sessions to
 ecosystem and other federated servers.
 
 #### Local sessions only
-**NOTE:** This does not affect 3PID lookups (queries to find Matrix IDs) which will remain public due to limitation
-in the Matrix protocol.
+**NOTE:** This does not affect 3PID lookups (queries to find Matrix IDs). See [Federation](../../features/federation.md)
+to disable remote lookup for those.
 
 This configuration ensures maximum confidentiality and privacy.
 Typical use cases:
@@ -256,37 +202,22 @@ No 3PID will be sent to a remote Identity server and all validation will be perf
 On the flip side, people with *Remote* 3PID scopes will not be found from other servers.
 
 Use the following values:
-```
-session:
-  policy:
-    validation:
-      enabled: true
-      forLocal:
+```yaml
+session.policy.validation.enabled: true
+session.policy.validation.forLocal:
   enabled: true
   toLocal: true
   toRemote:
     enabled: false
-      forRemote:
+session.policy.validation.forRemote:
   enabled: true
   toLocal: true
   toRemote:
     enabled: false
 ```
 
-**IMPORTANT**: When using local-only mode, you will also need to link mxisd to synapse if you want user searches and invites to work.
-To do so, add/edit the following configuration keys:
-```
-sql:
-  enabled: true
-  type: 'postgresql'
-  connection: ''
-```
-- `sql.enabled` set to `true` to activate the SQL backend.
-- `sql.type` can be set to `sqlite` or `postgresql`, depending on your synapse setup.
-- `sql.connection` use a JDBC format which is appened after the `jdbc:type:` connection URI.
-Example values for each type:
-  - `sqlite`: `/path/to/homeserver.db`
-  - `postgresql`: `//localhost/database?user=synapse&password=synapse`
+**IMPORTANT**: When using local-only mode and if you are using synapse, you will also need to enable its dedicated Identity
+store if you want user searches and invites to work. To do so, see the [dedicated document](../../stores/synapse.md).
 
 #### Remote sessions only
 This configuration ensures all 3PID are made public for maximum compatibility and reach within the Matrix ecosystem, at
@@ -297,17 +228,14 @@ Typical use cases:
 - Homeserver with registration enabled
 
 Use the following values:
-```
-session:
-  policy:
-    validation:
-      enabled: true
-      forLocal:
+```yaml
+session.policy.validation.enabled: true
+session.policy.validation.forLocal:
   enabled: true
   toLocal: false
   toRemote:
     enabled: true
-      forRemote:
+session.policy.validation.forRemote:
   enabled: true
   toLocal: false
   toRemote:
@@ -318,10 +246,7 @@ session:
 This configuration would disable 3PID session altogether, preventing users from adding emails and/or phone numbers to
 their profiles.  
 This would be used if mxisd is also performing authentication for the Homeserver, typically with synapse and the
-[REST Auth module](https://github.com/kamax-io/matrix-synapse-rest-auth).
-
-While this feature is not yet ready in the REST auth module, you would use this configuration mode to auto-populate 3PID
-at user login and prevent any further add.
+[REST password provider](https://github.com/kamax-io/matrix-synapse-rest-auth).
 
 **This mode comes with several important restrictions:**
 - This does not prevent users from removing 3PID from their profile. They would be unable to add them back!
@@ -330,9 +255,6 @@ at user login and prevent any further add.
 It is therefore recommended to not fully disable sessions but instead restrict specific set of 3PID and Session scopes.
 
 Use the following values to enable this mode:
-```
-session:
-  policy:
-    validation:
-      enabled: false
+```yaml
+session.policy.validation.enabled: false
 ```

src/docker/start.sh

@@ -1,2 +1,25 @@
-#!/bin/sh
+#!/usr/bin/env bash
+if [[ -n "$CONF_FILE_PATH" ]] && [ ! -f "$CONF_FILE_PATH" ]; then
+    echo "Generating config file $CONF_FILE_PATH"
+    touch "CONF_FILE_PATH"
+
+    if [[ -n "$MATRIX_DOMAIN" ]]; then
+        echo "Setting matrix domain to $MATRIX_DOMAIN"
+        echo "matrix.domain: $MATRIX_DOMAIN" >> "$CONF_FILE_PATH"
+    fi
+
+    if [[ -n "$SIGN_KEY_PATH" ]]; then
+        echo "Setting signing key path to $SIGN_KEY_PATH"
+        echo "key.path: $SIGN_KEY_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    if [[ -n "$SQLITE_DATABASE_PATH" ]]; then
+        echo "Setting SQLite DB path to $SQLITE_DATABASE_PATH"
+        echo "storage.provider.sqlite.database: $SQLITE_DATABASE_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    echo "Starting mxisd..."
+    echo
+fi
+
 exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.config.location=/etc/mxisd/ -Dspring.config.name=mxisd -jar /mxisd.jar

src/main/java/io/kamax/mxisd/auth/AuthManager.java

@@ -21,8 +21,9 @@
 package io.kamax.mxisd.auth;
 
 import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
 import io.kamax.matrix._MatrixID;
-import io.kamax.mxisd.ThreePid;
+import io.kamax.matrix._ThreePid;
 import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
@@ -42,8 +43,7 @@ public class AuthManager {
 
     private Logger log = LoggerFactory.getLogger(AuthManager.class);
 
-    @Autowired
-    private List<AuthenticatorProvider> providers = new ArrayList<>();
+    private List<AuthenticatorProvider> providers;
 
     @Autowired
     private MatrixConfig mxCfg;
@@ -51,28 +51,33 @@ public class AuthManager {
     @Autowired
     private InvitationManager invMgr;
 
+    public AuthManager(List<AuthenticatorProvider> providers) {
+        this.providers = new ArrayList<>(providers);
+    }
+
     public UserAuthResult authenticate(String id, String password) {
-        _MatrixID mxid = new MatrixID(id);
+        _MatrixID mxid = MatrixID.asAcceptable(id);
         for (AuthenticatorProvider provider : providers) {
             if (!provider.isEnabled()) {
                 continue;
             }
 
+            log.info("Attempting auth with " + provider.getClass().getSimpleName());
             BackendAuthResult result = provider.authenticate(mxid, password);
             if (result.isSuccess()) {
 
                 String mxId;
                 if (UserIdType.Localpart.is(result.getId().getType())) {
-                    mxId = new MatrixID(result.getId().getValue(), mxCfg.getDomain()).getId();
+                    mxId = MatrixID.from(result.getId().getValue(), mxCfg.getDomain()).acceptable().getId();
                 } else if (UserIdType.MatrixID.is(result.getId().getType())) {
-                    mxId = new MatrixID(result.getId().getValue()).getId();
+                    mxId = MatrixID.asAcceptable(result.getId().getValue()).getId();
                 } else {
                     log.warn("Unsupported User ID type {} for backend {}", result.getId().getType(), provider.getClass().getSimpleName());
                     continue;
                 }
 
                 UserAuthResult authResult = new UserAuthResult().success(result.getProfile().getDisplayName());
-                for (ThreePid pid : result.getProfile().getThreePids()) {
+                for (_ThreePid pid : result.getProfile().getThreePids()) {
                     authResult.withThreePid(pid.getMedium(), pid.getAddress());
                 }
                 log.info("{} was authenticated by {}, publishing 3PID mappings, if any", id, provider.getClass().getSimpleName());

src/main/java/io/kamax/mxisd/auth/UserAuthResult.java

@@ -20,7 +20,7 @@
 
 package io.kamax.mxisd.auth;
 
-import io.kamax.mxisd.ThreePid;
+import io.kamax.matrix.ThreePid;
 
 import java.util.Collections;
 import java.util.HashSet;

src/main/java/io/kamax/mxisd/auth/provider/BackendAuthResult.java

@@ -20,7 +20,7 @@
 
 package io.kamax.mxisd.auth.provider;
 
-import io.kamax.mxisd.ThreePid;
+import io.kamax.matrix.ThreePid;
 import io.kamax.mxisd.UserID;
 import io.kamax.mxisd.UserIdType;
 
@@ -49,8 +49,9 @@ public class BackendAuthResult {
         return r;
     }
 
-    public void fail() {
+    public BackendAuthResult fail() {
         success = false;
+        return this;
     }
 
     public static BackendAuthResult success(String id, UserIdType type, String displayName) {
@@ -63,11 +64,11 @@ public class BackendAuthResult {
         return r;
     }
 
-    public void succeed(String id, String type, String displayName) {
+    public BackendAuthResult succeed(String id, String type, String displayName) {
         this.success = true;
         this.id = new UserID(type, id);
-        this.profile = new BackendAuthProfile();
         this.profile.displayName = displayName;
+        return this;
     }
 
     private Boolean success;

src/main/java/io/kamax/mxisd/backend/firebase/GoogleFirebaseAuthenticator.java

@@ -23,9 +23,9 @@ package io.kamax.mxisd.backend.firebase;
 import com.google.firebase.auth.UserInfo;
 import com.google.i18n.phonenumbers.NumberParseException;
 import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import io.kamax.matrix.ThreePid;
 import io.kamax.matrix.ThreePidMedium;
 import io.kamax.matrix._MatrixID;
-import io.kamax.mxisd.ThreePid;
 import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;

src/main/java/io/kamax/mxisd/backend/google/GoogleProviderBackend.java

@@ -0,0 +1,134 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.google;
+
+import com.google.api.client.googleapis.auth.oauth2.GoogleIdToken;
+import com.google.api.client.googleapis.auth.oauth2.GoogleIdTokenVerifier;
+import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
+import com.google.api.client.http.HttpTransport;
+import com.google.api.client.json.JsonFactory;
+import com.google.api.client.json.jackson2.JacksonFactory;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import io.kamax.mxisd.config.GoogleConfig;
+import io.kamax.mxisd.lookup.strategy.LookupStrategy;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.io.IOException;
+import java.security.GeneralSecurityException;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Optional;
+
+@Component
+public class GoogleProviderBackend implements AuthenticatorProvider {
+
+    private final Logger log = LoggerFactory.getLogger(GoogleProviderBackend.class);
+    private final GoogleConfig cfg;
+    private final LookupStrategy lookup;
+
+    private GoogleIdTokenVerifier verifier;
+
+    public Optional<GoogleIdToken> extractToken(String data) throws GeneralSecurityException, IOException {
+        return Optional.ofNullable(verifier.verify(data));
+    }
+
+    public List<ThreePid> extractThreepids(GoogleIdToken token) {
+        List<ThreePid> tpids = new ArrayList<>();
+        tpids.add(new ThreePid("io.kamax.google.id", token.getPayload().getSubject()));
+        if (token.getPayload().getEmailVerified()) {
+            tpids.add(new ThreePid("email", token.getPayload().getEmail()));
+        }
+        return tpids;
+    }
+
+    @Autowired
+    public GoogleProviderBackend(GoogleConfig cfg, LookupStrategy lookup) {
+        this.cfg = cfg;
+        this.lookup = lookup;
+
+        if (isEnabled()) {
+            try {
+                HttpTransport transport = GoogleNetHttpTransport.newTrustedTransport();
+                JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
+
+                verifier = new GoogleIdTokenVerifier.Builder(transport, jsonFactory)
+                        .setAudience(Collections.singletonList(cfg.getClient().getId()))
+                        .build();
+            } catch (IOException | GeneralSecurityException e) {
+                throw new RuntimeException(e);
+            }
+        }
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return cfg.isEnabled();
+    }
+
+    @Override
+    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
+        BackendAuthResult result = new BackendAuthResult();
+
+        try {
+            return extractToken(password).map(idToken -> {
+                GoogleIdToken.Payload payload = idToken.getPayload();
+                if (!payload.getEmailVerified()) { // We only want users who validated their email
+                    return BackendAuthResult.failure();
+                }
+
+                // Get user identifier
+                String userId = payload.getSubject();
+
+                // We validate that the user who authenticated has his Google account associated already
+                return lookup.find("io.kamax.google.id", userId, false).map(r -> {
+
+                    if (!r.getMxid().equals(mxid)) {
+                        return result.fail();
+                    }
+
+                    // Get profile information from payload
+                    extractThreepids(idToken).forEach(result::withThreePid);
+                    String name = (String) payload.get("name");
+
+                    payload.getUnknownKeys().keySet().forEach(key -> {
+                        log.info("Unknown key in Google profile: {} -> ", key, payload.get(key));
+                    });
+
+                    return result.succeed(mxid.getId(), UserIdType.MatrixID.getId(), name);
+                }).orElse(BackendAuthResult.failure());
+            }).orElse(BackendAuthResult.failure());
+        } catch (GeneralSecurityException e) {
+            throw new RuntimeException(e);
+        } catch (IOException e) {
+            log.error("Unable to authenticate via Google due to network error", e);
+            return result.fail();
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java

@@ -20,12 +20,17 @@
 
 package io.kamax.mxisd.backend.ldap;
 
+import com.google.i18n.phonenumbers.NumberParseException;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix.ThreePidMedium;
 import io.kamax.matrix._MatrixID;
 import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.MatrixConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
+import io.kamax.mxisd.util.GsonUtil;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
@@ -41,12 +46,18 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;
 
 import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
 
 @Component
-public class LdapAuthProvider extends LdapGenericBackend implements AuthenticatorProvider {
+public class LdapAuthProvider extends LdapBackend implements AuthenticatorProvider {
 
     private Logger log = LoggerFactory.getLogger(LdapAuthProvider.class);
 
+    private PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+
     @Autowired
     public LdapAuthProvider(LdapConfig cfg, MatrixConfig mxCfg) {
         super(cfg, mxCfg);
@@ -57,6 +68,21 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
         return getCfg().isEnabled();
     }
 
+    private Optional<String> getMsisdn(String phoneNumber) {
+        try { // FIXME export into dedicated ThreePid class within SDK (copy from Firebase Auth)
+            return Optional.of(phoneUtil.format(
+                    phoneUtil.parse(
+                            phoneNumber,
+                            null // No default region
+                    ),
+                    PhoneNumberUtil.PhoneNumberFormat.E164
+            ).substring(1)); // We want without the leading +
+        } catch (NumberParseException e) {
+            log.warn("Invalid phone number: {}", phoneNumber);
+            return Optional.empty();
+        }
+    }
+
     @Override
     public BackendAuthResult authenticate(_MatrixID mxid, String password) {
         log.info("Performing auth for {}", mxid);
@@ -66,7 +92,7 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
             bind(conn);
 
             String uidType = getAt().getUid().getType();
-            String userFilterValue = StringUtils.equals(LdapGenericBackend.UID, uidType) ? mxid.getLocalPart() : mxid.getId();
+            String userFilterValue = StringUtils.equals(LdapBackend.UID, uidType) ? mxid.getLocalPart() : mxid.getId();
             if (StringUtils.isBlank(userFilterValue)) {
                 log.warn("Username is empty, failing auth");
                 return BackendAuthResult.failure();
@@ -74,7 +100,19 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
 
             String userFilter = "(" + getUidAtt() + "=" + userFilterValue + ")";
             userFilter = buildWithFilter(userFilter, getCfg().getAuth().getFilter());
-            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, getUidAtt(), getAt().getName())) {
+
+            Set<String> attributes = new HashSet<>();
+            attributes.add(getUidAtt());
+            attributes.add(getAt().getName());
+            getAt().getThreepid().forEach((k, v) -> attributes.addAll(v));
+            String[] attArray = new String[attributes.size()];
+            attributes.toArray(attArray);
+
+            log.debug("Base DN: {}", getBaseDn());
+            log.debug("Query: {}", userFilter);
+            log.debug("Attributes: {}", GsonUtil.build().toJson(attArray));
+
+            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, attArray)) {
                 while (cursor.next()) {
                     Entry entry = cursor.get();
                     String dn = entry.getDn().getName();
@@ -99,7 +137,24 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
                     log.info("DN {} is a valid match", dn);
 
                     // TODO should we canonicalize the MXID?
-                    return BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                    BackendAuthResult result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                    log.info("Processing 3PIDs for profile");
+                    getAt().getThreepid().forEach((k, v) -> {
+                        log.info("Processing 3PID type {}", k);
+                        v.forEach(attId -> {
+                            List<String> values = getAttributes(entry, attId);
+                            log.info("\tAttribute {} has {} value(s)", attId, values.size());
+                            getAttributes(entry, attId).forEach(tpidValue -> {
+                                if (ThreePidMedium.PhoneNumber.is(k)) {
+                                    tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
+                                }
+                                result.withThreePid(new ThreePid(k, tpidValue));
+                            });
+                        });
+                    });
+
+                    log.info("Found {} 3PIDs", result.getProfile().getThreePids().size());
+                    return result;
                 }
             } catch (CursorLdapReferralException e) {
                 log.warn("Entity for {} is only available via referral, skipping", mxid);

src/main/java/io/kamax/mxisd/backend/ldap/LdapBackend.java

@@ -21,10 +21,10 @@
 package io.kamax.mxisd.backend.ldap;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.ldap.LdapAttributeConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.entry.Attribute;
+import org.apache.directory.api.ldap.model.entry.AttributeUtils;
 import org.apache.directory.api.ldap.model.entry.Entry;
 import org.apache.directory.api.ldap.model.exception.LdapException;
 import org.apache.directory.ldap.client.api.LdapConnection;
@@ -32,21 +32,24 @@ import org.apache.directory.ldap.client.api.LdapNetworkConnection;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Optional;
 
-public abstract class LdapGenericBackend {
+public abstract class LdapBackend {
 
     public static final String UID = "uid";
     public static final String MATRIX_ID = "mxid";
 
-    private Logger log = LoggerFactory.getLogger(LdapGenericBackend.class);
+    private Logger log = LoggerFactory.getLogger(LdapBackend.class);
 
     private LdapConfig cfg;
     private MatrixConfig mxCfg;
 
-    public LdapGenericBackend(LdapConfig cfg, MatrixConfig mxCfg) {
+    public LdapBackend(LdapConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.mxCfg = mxCfg;
     }
@@ -56,10 +59,10 @@ public abstract class LdapGenericBackend {
     }
 
     protected String getBaseDn() {
-        return cfg.getConn().getBaseDn();
+        return cfg.getConnection().getBaseDn();
     }
 
-    protected LdapAttributeConfig getAt() {
+    protected LdapConfig.Attribute getAt() {
         return cfg.getAttribute();
     }
 
@@ -68,14 +71,14 @@ public abstract class LdapGenericBackend {
     }
 
     protected synchronized LdapConnection getConn() throws LdapException {
-        return new LdapNetworkConnection(cfg.getConn().getHost(), cfg.getConn().getPort(), cfg.getConn().isTls());
+        return new LdapNetworkConnection(cfg.getConnection().getHost(), cfg.getConnection().getPort(), cfg.getConnection().isTls());
     }
 
     protected void bind(LdapConnection conn) throws LdapException {
-        if (StringUtils.isBlank(cfg.getConn().getBindDn()) && StringUtils.isBlank(cfg.getConn().getBindPassword())) {
+        if (StringUtils.isBlank(cfg.getConnection().getBindDn()) && StringUtils.isBlank(cfg.getConnection().getBindPassword())) {
             conn.anonymousBind();
         } else {
-            conn.bind(cfg.getConn().getBindDn(), cfg.getConn().getBindPassword());
+            conn.bind(cfg.getConnection().getBindDn(), cfg.getConnection().getBindPassword());
         }
     }
 
@@ -124,7 +127,6 @@ public abstract class LdapGenericBackend {
     public Optional<String> getAttribute(Entry entry, String attName) {
         Attribute attribute = entry.get(attName);
         if (attribute == null) {
-            log.info("DN {}: no attribute {}, skipping", entry.getDn(), attName);
             return Optional.empty();
         }
 
@@ -137,4 +139,22 @@ public abstract class LdapGenericBackend {
         return Optional.of(value);
     }
 
+    public List<String> getAttributes(Entry entry, String attName) {
+        List<String> values = new ArrayList<>();
+        javax.naming.directory.Attribute att = AttributeUtils.toAttributes(entry).get(attName);
+        if (att == null) {
+            return values;
+        }
+
+        try {
+            NamingEnumeration<?> list = att.getAll();
+            while (list.hasMore()) {
+                values.add(list.next().toString());
+            }
+        } catch (NamingException e) {
+            log.warn("Error while processing LDAP attribute {}, result could be incomplete!", attName, e);
+        }
+        return values;
+    }
+
 }

src/main/java/io/kamax/mxisd/backend/ldap/LdapDirectoryProvider.java

@@ -21,11 +21,11 @@
 package io.kamax.mxisd.backend.ldap;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.ldap.LdapAttributeConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.IDirectoryProvider;
 import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.util.GsonUtil;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
 import org.apache.directory.api.ldap.model.cursor.EntryCursor;
@@ -43,7 +43,7 @@ import java.util.ArrayList;
 import java.util.List;
 
 @Component
-public class LdapDirectoryProvider extends LdapGenericBackend implements IDirectoryProvider {
+public class LdapDirectoryProvider extends LdapBackend implements IDirectoryProvider {
 
     private Logger log = LoggerFactory.getLogger(LdapDirectoryProvider.class);
 
@@ -64,15 +64,18 @@ public class LdapDirectoryProvider extends LdapGenericBackend implements IDirect
         try (LdapConnection conn = getConn()) {
             bind(conn);
 
-            LdapAttributeConfig atCfg = getCfg().getAttribute();
+            LdapConfig.Attribute atCfg = getCfg().getAttribute();
 
             attributes = new ArrayList<>(attributes);
             attributes.add(getUidAtt());
             String[] attArray = new String[attributes.size()];
             attributes.toArray(attArray);
-
             String searchQuery = buildOrQueryWithFilter(getCfg().getDirectory().getFilter(), "*" + query + "*", attArray);
+
+            log.debug("Base DN: {}", getBaseDn());
             log.debug("Query: {}", searchQuery);
+            log.debug("Attributes: {}", GsonUtil.build().toJson(attArray));
+
             try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, attArray)) {
                 while (cursor.next()) {
                     Entry entry = cursor.get();

src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java

@@ -27,6 +27,7 @@ import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import io.kamax.mxisd.util.GsonUtil;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
 import org.apache.directory.api.ldap.model.cursor.EntryCursor;
@@ -44,7 +45,7 @@ import java.util.List;
 import java.util.Optional;
 
 @Component
-public class LdapThreePidProvider extends LdapGenericBackend implements IThreePidProvider {
+public class LdapThreePidProvider extends LdapBackend implements IThreePidProvider {
 
     private Logger log = LoggerFactory.getLogger(LdapThreePidProvider.class);
 
@@ -68,13 +69,20 @@ public class LdapThreePidProvider extends LdapGenericBackend implements IThreePi
     }
 
     private Optional<String> lookup(LdapConnection conn, String medium, String value) {
-        Optional<String> queryOpt = getCfg().getIdentity().getQuery(medium);
-        if (!queryOpt.isPresent()) {
+        Optional<String> tPidQueryOpt = getCfg().getIdentity().getQuery(medium);
+        if (!tPidQueryOpt.isPresent()) {
             log.warn("{} is not a configured 3PID type for LDAP lookup", medium);
             return Optional.empty();
         }
 
-        String searchQuery = queryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        // we merge 3PID specific query with global/specific filter, if one exists.
+        String tPidQuery = tPidQueryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        String searchQuery = buildWithFilter(tPidQuery, getCfg().getIdentity().getFilter());
+
+        log.debug("Base DN: {}", getBaseDn());
+        log.debug("Query: {}", searchQuery);
+        log.debug("Attributes: {}", GsonUtil.build().toJson(getUidAtt()));
+
         try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, getUidAtt())) {
             while (cursor.next()) {
                 Entry entry = cursor.get();

src/main/java/io/kamax/mxisd/backend/ldap/netiq/NetIqLdapAuthProvider.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.ldap.netiq;
+
+import io.kamax.mxisd.backend.ldap.LdapAuthProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.netiq.NetIqLdapConfig;
+import org.springframework.stereotype.Component;
+
+@Component
+public class NetIqLdapAuthProvider extends LdapAuthProvider {
+
+    public NetIqLdapAuthProvider(NetIqLdapConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    // FIXME this is duplicated in the other NetIQ classes, due to the Matrix ID generation code that was not abstracted
+    @Override
+    public String buildMatrixIdFromUid(String uid) {
+        return super.buildMatrixIdFromUid(uid).toLowerCase();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/ldap/netiq/NetIqLdapDirectoryProvider.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.ldap.netiq;
+
+import io.kamax.mxisd.backend.ldap.LdapDirectoryProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.netiq.NetIqLdapConfig;
+import org.springframework.stereotype.Component;
+
+@Component
+public class NetIqLdapDirectoryProvider extends LdapDirectoryProvider {
+
+    public NetIqLdapDirectoryProvider(NetIqLdapConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    // FIXME this is duplicated in the other NetIQ classes, due to the Matrix ID generation code that was not abstracted
+    @Override
+    public String buildMatrixIdFromUid(String uid) {
+        return super.buildMatrixIdFromUid(uid).toLowerCase();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/ldap/netiq/NetIqLdapThreePidProvider.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.ldap.netiq;
+
+import io.kamax.mxisd.backend.ldap.LdapThreePidProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.netiq.NetIqLdapConfig;
+import org.springframework.stereotype.Component;
+
+@Component
+public class NetIqLdapThreePidProvider extends LdapThreePidProvider {
+
+    public NetIqLdapThreePidProvider(NetIqLdapConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    // FIXME this is duplicated in the other NetIQ classes, due to the Matrix ID generation code that was not abstracted
+    @Override
+    public String buildMatrixIdFromUid(String uid) {
+        return super.buildMatrixIdFromUid(uid).toLowerCase();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/memory/MemoryIdentityStore.java

@@ -0,0 +1,173 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.memory;
+
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._ThreePid;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.memory.MemoryIdentityConfig;
+import io.kamax.mxisd.config.memory.MemoryStoreConfig;
+import io.kamax.mxisd.config.memory.MemoryThreePid;
+import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
+import io.kamax.mxisd.directory.IDirectoryProvider;
+import io.kamax.mxisd.lookup.SingleLookupReply;
+import io.kamax.mxisd.lookup.SingleLookupRequest;
+import io.kamax.mxisd.lookup.ThreePidMapping;
+import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import io.kamax.mxisd.profile.ProfileProvider;
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Optional;
+import java.util.function.Function;
+import java.util.function.Predicate;
+
+@Component
+public class MemoryIdentityStore implements AuthenticatorProvider, IDirectoryProvider, IThreePidProvider, ProfileProvider {
+
+    private final Logger logger = LoggerFactory.getLogger(MemoryIdentityStore.class);
+
+    private final MatrixConfig mxCfg;
+    private final MemoryStoreConfig cfg;
+
+    @Autowired
+    public MemoryIdentityStore(MatrixConfig mxCfg, MemoryStoreConfig cfg) {
+        this.mxCfg = mxCfg;
+        this.cfg = cfg;
+    }
+
+    public Optional<MemoryIdentityConfig> findByUsername(String username) {
+        return cfg.getIdentities().stream()
+                .filter(id -> StringUtils.equals(id.getUsername(), username))
+                .findFirst();
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return cfg.isEnabled();
+    }
+
+    private UserDirectorySearchResult search(
+            Predicate<MemoryIdentityConfig> predicate,
+            Function<MemoryIdentityConfig, UserDirectorySearchResult.Result> mapper
+    ) {
+        UserDirectorySearchResult search = new UserDirectorySearchResult();
+        cfg.getIdentities().stream().filter(predicate).map(mapper).forEach(search::addResult);
+        return search;
+    }
+
+    @Override
+    public UserDirectorySearchResult searchByDisplayName(String query) {
+        return search(
+                entry -> StringUtils.containsIgnoreCase(entry.getUsername(), query),
+                entry -> {
+                    UserDirectorySearchResult.Result result = new UserDirectorySearchResult.Result();
+                    result.setUserId(MatrixID.from(entry.getUsername(), mxCfg.getDomain()).acceptable().getId());
+                    result.setDisplayName(entry.getUsername());
+                    return result;
+                }
+        );
+    }
+
+    @Override
+    public UserDirectorySearchResult searchBy3pid(String query) {
+        return search(
+                entry -> entry.getThreepids().stream()
+                        .anyMatch(tpid -> StringUtils.containsIgnoreCase(tpid.getAddress(), query)),
+                entry -> {
+                    UserDirectorySearchResult.Result result = new UserDirectorySearchResult.Result();
+                    result.setUserId(MatrixID.from(entry.getUsername(), mxCfg.getDomain()).acceptable().getId());
+                    result.setDisplayName(entry.getUsername());
+                    return result;
+                }
+        );
+    }
+
+    @Override
+    public List<_ThreePid> getThreepids(_MatrixID mxid) {
+        List<_ThreePid> l = new ArrayList<>();
+        findByUsername(mxid.getLocalPart()).ifPresent(c -> l.addAll(c.getThreepids()));
+        return l;
+    }
+
+    @Override
+    public List<String> getRoles(_MatrixID mxid) {
+        List<String> l = new ArrayList<>();
+        findByUsername(mxid.getLocalPart()).ifPresent(c -> l.addAll(c.getRoles()));
+        return l;
+    }
+
+    @Override
+    public boolean isLocal() {
+        return true;
+    }
+
+    @Override
+    public int getPriority() {
+        return Integer.MAX_VALUE;
+    }
+
+    @Override
+    public Optional<SingleLookupReply> find(SingleLookupRequest request) {
+        logger.info("Performing lookup {} of type {}", request.getThreePid(), request.getType());
+        ThreePid req = new ThreePid(request.getType(), request.getThreePid());
+        for (MemoryIdentityConfig id : cfg.getIdentities()) {
+            for (MemoryThreePid threepid : id.getThreepids()) {
+                if (req.equals(new ThreePid(threepid.getMedium(), threepid.getAddress()))) {
+                    return Optional.of(new SingleLookupReply(request, new MatrixID(id.getUsername(), mxCfg.getDomain())));
+                }
+            }
+        }
+
+        return Optional.empty();
+    }
+
+    @Override
+    public List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
+        return Collections.emptyList();
+    }
+
+    @Override
+    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
+        return findByUsername(mxid.getLocalPart()).map(id -> {
+            if (!StringUtils.equals(id.getUsername(), mxid.getLocalPart())) {
+                return BackendAuthResult.failure();
+            } else {
+                BackendAuthResult result = new BackendAuthResult();
+                id.getThreepids().forEach(tpid -> result.withThreePid(new ThreePid(tpid.getMedium(), tpid.getAddress())));
+                result.succeed(mxid.getId(), UserIdType.MatrixID.getId(), "");
+                return result;
+            }
+        }).orElseGet(BackendAuthResult::failure);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/rest/RestDirectoryProvider.java

@@ -31,10 +31,12 @@ import io.kamax.mxisd.util.RestClientUtils;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang.StringUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
+import org.springframework.stereotype.Component;
 
 import java.io.IOException;
 import java.nio.charset.StandardCharsets;
 
+@Component
 public class RestDirectoryProvider extends RestProvider implements IDirectoryProvider {
 
     private MatrixConfig mxCfg;

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlAuthProvider.java

@@ -24,7 +24,7 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.ServerConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.invitation.InvitationManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -32,15 +32,15 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;
 
 @Component
-public class SqlAuthProvider implements AuthenticatorProvider {
+public class GenericSqlAuthProvider implements AuthenticatorProvider {
 
-    private Logger log = LoggerFactory.getLogger(SqlAuthProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlAuthProvider.class);
 
     @Autowired
     private ServerConfig srvCfg;
 
     @Autowired
-    private SqlProviderConfig cfg;
+    private GenericSqlProviderConfig cfg;
 
     @Autowired
     private InvitationManager invMgr;

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlDirectoryProvider.java

@@ -22,8 +22,8 @@ package io.kamax.mxisd.backend.sql;
 
 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.SqlConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.IDirectoryProvider;
 import io.kamax.mxisd.exception.InternalServerError;
@@ -39,16 +39,16 @@ import java.util.Optional;
 
 import static io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult.Result;
 
-public abstract class SqlDirectoryProvider implements IDirectoryProvider {
+public abstract class GenericSqlDirectoryProvider implements IDirectoryProvider {
 
-    private Logger log = LoggerFactory.getLogger(SqlDirectoryProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlDirectoryProvider.class);
 
     protected SqlConfig cfg;
     private MatrixConfig mxCfg;
 
     private SqlConnectionPool pool;
 
-    public SqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
+    public GenericSqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.pool = new SqlConnectionPool(cfg);
         this.mxCfg = mxCfg;
@@ -72,7 +72,7 @@ public abstract class SqlDirectoryProvider implements IDirectoryProvider {
         return Optional.of(item);
     }
 
-    public UserDirectorySearchResult search(String searchTerm, SqlProviderConfig.Query query) {
+    public UserDirectorySearchResult search(String searchTerm, GenericSqlProviderConfig.Query query) {
         try (Connection conn = pool.get()) {
             log.info("Will execute query: {}", query.getValue());
             try (PreparedStatement stmt = conn.prepareStatement(query.getValue())) {

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlThreePidProvider.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class GenericSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public GenericSqlThreePidProvider(GenericSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/sql/SqlThreePidProvider.java

@@ -21,43 +21,48 @@
 package io.kamax.mxisd.backend.sql;
 
 import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._ThreePid;
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.SqlConfig;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import io.kamax.mxisd.profile.ProfileProvider;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;
 
 import java.sql.Connection;
 import java.sql.PreparedStatement;
 import java.sql.ResultSet;
 import java.sql.SQLException;
 import java.util.ArrayList;
+import java.util.Collections;
 import java.util.List;
 import java.util.Optional;
 
-@Component
-public class SqlThreePidProvider implements IThreePidProvider {
+public abstract class SqlThreePidProvider implements IThreePidProvider, ProfileProvider {
 
     private Logger log = LoggerFactory.getLogger(SqlThreePidProvider.class);
 
-    private SqlProviderConfig cfg;
+    private SqlConfig cfg;
     private MatrixConfig mxCfg;
 
     private SqlConnectionPool pool;
 
-    @Autowired
-    public SqlThreePidProvider(SqlProviderConfig cfg, MatrixConfig mxCfg) {
+    public SqlThreePidProvider(SqlConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.pool = new SqlConnectionPool(cfg);
         this.mxCfg = mxCfg;
     }
 
+    protected Connection getConnection() throws SQLException {
+        return pool.get();
+    }
+
     @Override
     public boolean isEnabled() {
         return cfg.isEnabled();
@@ -113,4 +118,31 @@ public class SqlThreePidProvider implements IThreePidProvider {
         return new ArrayList<>();
     }
 
+    @Override
+    public List<_ThreePid> getThreepids(_MatrixID mxid) {
+        List<_ThreePid> threepids = new ArrayList<>();
+
+        String stmtSql = cfg.getProfile().getThreepid().getQuery();
+        try (Connection conn = getConnection()) {
+            PreparedStatement stmt = conn.prepareStatement(stmtSql);
+            stmt.setString(1, mxid.getId());
+
+            ResultSet rSet = stmt.executeQuery();
+            while (rSet.next()) {
+                String medium = rSet.getString("medium");
+                String address = rSet.getString("address");
+                threepids.add(new ThreePid(medium, address));
+            }
+
+            return threepids;
+        } catch (SQLException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    @Override
+    public List<String> getRoles(_MatrixID mxid) {
+        return Collections.emptyList();
+    }
+
 }

src/main/java/io/kamax/mxisd/backend/sql/SynapseSqlDirectoryProvider.java

@@ -21,7 +21,7 @@
 package io.kamax.mxisd.backend.sql;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
 import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang.StringUtils;
@@ -32,17 +32,15 @@ import java.sql.PreparedStatement;
 import java.sql.SQLException;
 
 @Component
-public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
-
-    private SynapseSqlProviderConfig cfg;
+public class SynapseSqlDirectoryProvider extends GenericSqlDirectoryProvider {
 
     @Autowired
-    public SynapseSqliteDirectoryProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
+    public SynapseSqlDirectoryProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
         super(cfg, mxCfg);
 
         if (StringUtils.equals("sqlite", cfg.getType())) {
             String userId = "'@' || p.user_id || ':" + mxCfg.getDomain() + "'";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
             queries.getName().setValue(
                     "select " + userId + ", displayname from profiles p where displayname like ?");
             queries.getThreepid().setValue(
@@ -51,7 +49,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
                             "where t.address like ?");
         } else if (StringUtils.equals("postgresql", cfg.getType())) {
             String userId = "concat('@',p.user_id,':" + mxCfg.getDomain() + "')";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
             queries.getName().setValue(
                     "select " + userId + ", displayname from profiles p where displayname ilike ?");
             queries.getThreepid().setValue(

src/main/java/io/kamax/mxisd/backend/sql/SynapseSqlThreePidProvider.java

@@ -0,0 +1,69 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
+import io.kamax.mxisd.profile.ProfileWriter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.SQLException;
+import java.time.Instant;
+
+@Component
+public class SynapseSqlThreePidProvider extends SqlThreePidProvider implements ProfileWriter {
+
+    private final Logger log = LoggerFactory.getLogger(SynapseSqlThreePidProvider.class);
+
+    @Autowired
+    public SynapseSqlThreePidProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    @Override
+    public boolean addThreepid(_MatrixID mxid, ThreePid tpid) {
+        try (Connection conn = getConnection()) {
+            PreparedStatement stmt = conn.prepareStatement("INSERT INTO user_threepids (user_id, medium, address, validated_at, added_at) values (?,?,?,?,?)");
+            stmt.setString(1, mxid.getId());
+            stmt.setString(2, tpid.getMedium());
+            stmt.setString(3, tpid.getAddress());
+            stmt.setLong(4, Instant.now().toEpochMilli());
+            stmt.setLong(5, Instant.now().toEpochMilli());
+
+            int rows = stmt.executeUpdate();
+            if (rows != 1) {
+                log.error("Unable to update 3PID info. Modified row(s): {}", rows);
+            }
+
+            return true;
+        } catch (SQLException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/wordpress/WordpressAuthData.java

@@ -0,0 +1,62 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.wordpress;
+
+public class WordpressAuthData {
+
+    public String token;
+    private String userEmail;
+    private String userNicename;
+    private String userDisplayName;
+
+    public String getToken() {
+        return token;
+    }
+
+    public void setToken(String token) {
+        this.token = token;
+    }
+
+    public String getUserEmail() {
+        return userEmail;
+    }
+
+    public void setUserEmail(String userEmail) {
+        this.userEmail = userEmail;
+    }
+
+    public String getUserNicename() {
+        return userNicename;
+    }
+
+    public void setUserNicename(String userNicename) {
+        this.userNicename = userNicename;
+    }
+
+    public String getUserDisplayName() {
+        return userDisplayName;
+    }
+
+    public void setUserDisplayName(String userDisplayName) {
+        this.userDisplayName = userDisplayName;
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/wordpress/WordpressAuthProvider.java

@@ -0,0 +1,68 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.wordpress;
+
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class WordpressAuthProvider implements AuthenticatorProvider {
+
+    private final Logger log = LoggerFactory.getLogger(WordpressAuthProvider.class);
+
+    private WordpressRestBackend wordpress;
+
+    @Autowired
+    public WordpressAuthProvider(WordpressRestBackend wordpress) {
+        this.wordpress = wordpress;
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return wordpress.isEnabled();
+    }
+
+    @Override
+    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
+        try {
+            WordpressAuthData data = wordpress.authenticate(mxid.getLocalPart(), password);
+            BackendAuthResult result = new BackendAuthResult();
+            if (StringUtils.isNotBlank(data.getUserEmail())) {
+                result.withThreePid(new ThreePid("email", data.getUserEmail()));
+            }
+            result.succeed(mxid.getId(), UserIdType.MatrixID.getId(), data.getUserDisplayName());
+            return result;
+        } catch (IllegalArgumentException e) {
+            log.error("Authentication failed for {}: {}", mxid.getId(), e.getMessage());
+            return BackendAuthResult.failure();
+        }
+
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/wordpress/WordpressDirectoryProvider.java

@@ -0,0 +1,116 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.wordpress;
+
+import io.kamax.matrix.MatrixID;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.wordpress.WordpressConfig;
+import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
+import io.kamax.mxisd.directory.IDirectoryProvider;
+import io.kamax.mxisd.exception.InternalServerError;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.Optional;
+
+@Component
+public class WordpressDirectoryProvider implements IDirectoryProvider {
+
+    private final Logger log = LoggerFactory.getLogger(WordpressDirectoryProvider.class);
+
+    private WordpressConfig cfg;
+    private WordressSqlBackend wordpress;
+    private MatrixConfig mxCfg;
+
+    @Autowired
+    public WordpressDirectoryProvider(WordpressConfig cfg, WordressSqlBackend wordpress, MatrixConfig mxCfg) {
+        this.cfg = cfg;
+        this.wordpress = wordpress;
+        this.mxCfg = mxCfg;
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return wordpress.isEnabled();
+    }
+
+    protected void setParameters(PreparedStatement stmt, String searchTerm) throws SQLException {
+        for (int i = 1; i <= stmt.getParameterMetaData().getParameterCount(); i++) {
+            stmt.setString(i, "%" + searchTerm + "%");
+        }
+    }
+
+    protected Optional<UserDirectorySearchResult.Result> processRow(ResultSet rSet) throws SQLException {
+        UserDirectorySearchResult.Result item = new UserDirectorySearchResult.Result();
+        item.setUserId(rSet.getString(1));
+        item.setDisplayName(rSet.getString(2));
+        return Optional.of(item);
+    }
+
+    public UserDirectorySearchResult search(String searchTerm, String query) {
+        try (Connection conn = wordpress.getConnection()) {
+            log.info("Will execute query: {}", query);
+            try (PreparedStatement stmt = conn.prepareStatement(query)) {
+                setParameters(stmt, searchTerm);
+
+                try (ResultSet rSet = stmt.executeQuery()) {
+                    UserDirectorySearchResult result = new UserDirectorySearchResult();
+                    result.setLimited(false);
+
+                    while (rSet.next()) {
+                        processRow(rSet).ifPresent(e -> {
+                            try {
+                                e.setUserId(MatrixID.from(e.getUserId(), mxCfg.getDomain()).valid().getId());
+                                result.addResult(e);
+                            } catch (IllegalArgumentException ex) {
+                                log.warn("Ignoring result {} - Invalid characters for a Matrix ID", e.getUserId());
+                            }
+                        });
+                    }
+
+                    return result;
+                }
+            }
+        } catch (SQLException e) {
+            e.printStackTrace();
+            throw new InternalServerError(e);
+        }
+    }
+
+    @Override
+    public UserDirectorySearchResult searchByDisplayName(String searchTerm) {
+        log.info("Searching users by display name using '{}'", searchTerm);
+        return search(searchTerm, cfg.getSql().getQuery().getDirectory().get("name"));
+    }
+
+    @Override
+    public UserDirectorySearchResult searchBy3pid(String searchTerm) {
+        log.info("Searching users by 3PID using '{}'", searchTerm);
+        return search(searchTerm, cfg.getSql().getQuery().getDirectory().get("threepid"));
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/wordpress/WordpressRestBackend.java

@@ -0,0 +1,143 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.wordpress;
+
+import com.google.gson.JsonObject;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.matrix.json.InvalidJsonException;
+import io.kamax.mxisd.config.wordpress.WordpressConfig;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpRequestBase;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.io.IOException;
+
+@Component
+public class WordpressRestBackend {
+
+    private final Logger log = LoggerFactory.getLogger(WordpressRestBackend.class);
+    private final String jsonPath = "/wp-json";
+    private final String jwtPath = "/jwt-auth/v1";
+
+    private WordpressConfig cfg;
+    private CloseableHttpClient client;
+
+    private String jsonEndpoint;
+    private String jwtEndpoint;
+
+    private String token;
+
+    @Autowired
+    public WordpressRestBackend(WordpressConfig cfg, CloseableHttpClient client) {
+        this.cfg = cfg;
+        this.client = client;
+
+        if (!cfg.isEnabled()) {
+            return;
+        }
+
+        jsonEndpoint = cfg.getRest().getBase() + jsonPath;
+        jwtEndpoint = jsonEndpoint + jwtPath;
+        validateConfig();
+    }
+
+    private void validateConfig() {
+        log.info("Validating JWT auth endpoint");
+        try (CloseableHttpResponse res = client.execute(new HttpGet(jwtEndpoint))) {
+            int status = res.getStatusLine().getStatusCode();
+            if (status != 200) {
+                log.warn("JWT auth endpoint check failed: Got status code {}", status);
+                return;
+            }
+
+            String data = EntityUtils.toString(res.getEntity());
+            if (StringUtils.isBlank(data)) {
+                log.warn("JWT auth endpoint check failed: Got no/empty body data");
+            }
+
+            JsonObject body = GsonUtil.parseObj(data);
+            if (!body.has("namespace")) {
+                log.warn("JWT auth endpoint check failed: invalid namespace");
+            }
+
+            log.info("JWT auth endpoint check succeeded");
+        } catch (InvalidJsonException e) {
+            log.warn("JWT auth endpoint check failed: Invalid JSON response: {}", e.getMessage());
+        } catch (IOException e) {
+            log.warn("JWT auth endpoint check failed: Could not read API endpoint: {}", e.getMessage());
+        }
+    }
+
+    public boolean isEnabled() {
+        return cfg.isEnabled();
+    }
+
+    protected WordpressAuthData authenticate(String username, String password) {
+        JsonObject body = new JsonObject();
+        body.addProperty("username", username);
+        body.addProperty("password", password);
+        HttpPost req = RestClientUtils.post(jwtEndpoint + "/token", body);
+        try (CloseableHttpResponse res = client.execute(req)) {
+            int status = res.getStatusLine().getStatusCode();
+            String bodyRes = EntityUtils.toString(res.getEntity());
+            if (status != 200) {
+                throw new IllegalArgumentException(bodyRes);
+            }
+
+            return GsonUtil.get().fromJson(bodyRes, WordpressAuthData.class);
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    protected void authenticate() {
+        WordpressAuthData data = authenticate(
+                cfg.getRest().getCredential().getUsername(),
+                cfg.getRest().getCredential().getPassword());
+        log.info("Internal authentication: success, logged in as " + data.getUserNicename());
+        token = data.getToken();
+    }
+
+    protected CloseableHttpResponse runRequest(HttpRequestBase request) throws IOException {
+        request.setHeader("Authorization", "Bearer " + token);
+        return client.execute(request);
+    }
+
+    public CloseableHttpResponse withAuthentication(HttpRequestBase request) throws IOException {
+        CloseableHttpResponse response = runRequest(request);
+        if (response.getStatusLine().getStatusCode() == 403) { //FIXME we should check the JWT expiration time
+            authenticate();
+            response = runRequest(request);
+        }
+
+        return response;
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/wordpress/WordpressThreePidProvider.java

@@ -0,0 +1,118 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.wordpress;
+
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.wordpress.WordpressConfig;
+import io.kamax.mxisd.lookup.SingleLookupReply;
+import io.kamax.mxisd.lookup.SingleLookupRequest;
+import io.kamax.mxisd.lookup.ThreePidMapping;
+import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+
+@Component
+public class WordpressThreePidProvider implements IThreePidProvider {
+
+    private final Logger log = LoggerFactory.getLogger(WordpressThreePidProvider.class);
+
+    private MatrixConfig mxCfg;
+    private WordpressConfig cfg;
+    private WordressSqlBackend wordpress;
+
+    @Autowired
+    public WordpressThreePidProvider(MatrixConfig mxCfg, WordpressConfig cfg, WordressSqlBackend wordpress) {
+        this.mxCfg = mxCfg;
+        this.cfg = cfg;
+        this.wordpress = wordpress;
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return wordpress.isEnabled();
+    }
+
+    @Override
+    public boolean isLocal() {
+        return true;
+    }
+
+    @Override
+    public int getPriority() {
+        return 15;
+    }
+
+    protected Optional<_MatrixID> find(ThreePid tpid) {
+        String query = cfg.getSql().getQuery().getThreepid().get(tpid.getMedium());
+        if (Objects.isNull(query)) {
+            return Optional.empty();
+        }
+
+        try (Connection conn = wordpress.getConnection()) {
+            PreparedStatement stmt = conn.prepareStatement(query);
+            stmt.setString(1, tpid.getAddress());
+
+            try (ResultSet rSet = stmt.executeQuery()) {
+                while (rSet.next()) {
+                    String uid = rSet.getString("uid");
+                    log.info("Found match: {}", uid);
+                    try {
+                        return Optional.of(MatrixID.from(uid, mxCfg.getDomain()).valid());
+                    } catch (IllegalArgumentException ex) {
+                        log.warn("Ignoring match {} - Invalid characters for a Matrix ID", uid);
+                    }
+                }
+
+                log.info("No valid match found in Wordpress");
+                return Optional.empty();
+            }
+        } catch (SQLException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    @Override
+    public Optional<SingleLookupReply> find(SingleLookupRequest request) {
+        return find(new ThreePid(request.getType(), request.getThreePid())).map(mxid -> new SingleLookupReply(request, mxid));
+    }
+
+    @Override
+    public List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
+        for (ThreePidMapping tpidMap : mappings) {
+            find(new ThreePid(tpidMap.getMedium(), tpidMap.getValue())).ifPresent(mxid -> tpidMap.setMxid(mxid.getId()));
+        }
+        return mappings;
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/wordpress/WordressSqlBackend.java

@@ -0,0 +1,61 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.wordpress;
+
+import com.mchange.v2.c3p0.ComboPooledDataSource;
+import io.kamax.mxisd.config.wordpress.WordpressConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.sql.Connection;
+import java.sql.SQLException;
+
+@Component
+public class WordressSqlBackend {
+
+    private Logger log = LoggerFactory.getLogger(WordressSqlBackend.class);
+
+    private WordpressConfig cfg;
+
+    private ComboPooledDataSource ds;
+
+    @Autowired
+    public WordressSqlBackend(WordpressConfig cfg) {
+        this.cfg = cfg;
+
+        ds = new ComboPooledDataSource();
+        ds.setJdbcUrl("jdbc:" + cfg.getSql().getType() + ":" + cfg.getSql().getConnection());
+        ds.setMinPoolSize(1);
+        ds.setMaxPoolSize(10);
+        ds.setAcquireIncrement(2);
+    }
+
+    public boolean isEnabled() {
+        return cfg.isEnabled();
+    }
+
+    public Connection getConnection() throws SQLException {
+        return ds.getConnection();
+    }
+
+}

src/main/java/io/kamax/mxisd/config/DirectoryConfig.java

@@ -18,45 +18,42 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.config.ldap;
+package io.kamax.mxisd.config;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
 @Configuration
-@ConfigurationProperties(prefix = "ldap.attribute")
-public class LdapAttributeConfig {
+@ConfigurationProperties("directory")
+public class DirectoryConfig {
 
-    private LdapAttributeUidConfig uid;
-    private String name;
-    private Map<String, List<String>> threepid = new HashMap<>();
+    private final transient Logger log = LoggerFactory.getLogger(DnsOverwriteConfig.class);
 
-    public LdapAttributeUidConfig getUid() {
-        return uid;
+    public static class Exclude {
+
+        private boolean homeserver;
+
+        public boolean getHomeserver() {
+            return homeserver;
         }
 
-    public void setUid(LdapAttributeUidConfig uid) {
-        this.uid = uid;
+        public Exclude setHomeserver(boolean homeserver) {
+            this.homeserver = homeserver;
+            return this;
         }
 
-    public String getName() {
-        return name;
     }
 
-    public void setName(String name) {
-        this.name = name;
+    private Exclude exclude = new Exclude();
+
+    public Exclude getExclude() {
+        return exclude;
     }
 
-    public Map<String, List<String>> getThreepid() {
-        return threepid;
-    }
-
-    public void setThreepid(Map<String, List<String>> threepid) {
-        this.threepid = threepid;
+    public void setExclude(Exclude exclude) {
+        this.exclude = exclude;
     }
 
 }

src/main/java/io/kamax/mxisd/config/GoogleConfig.java

@@ -0,0 +1,107 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import javax.annotation.PostConstruct;
+
+@Configuration
+@ConfigurationProperties("google")
+public class GoogleConfig {
+
+    public static class Client {
+
+        private String id;
+        private String secret;
+
+        public String getId() {
+            return id;
+        }
+
+        public void setId(String id) {
+            this.id = id;
+        }
+
+        public String getSecret() {
+            return secret;
+        }
+
+        public void setSecret(String secret) {
+            this.secret = secret;
+        }
+
+    }
+
+    private final Logger log = LoggerFactory.getLogger(GoogleConfig.class);
+
+    private boolean enabled;
+    private Client client = new Client();
+    private String medium = "io.kamax.google.id";
+    private String prefix = "google_";
+
+    public boolean isEnabled() {
+        return enabled;
+    }
+
+    public void setEnabled(boolean enabled) {
+        this.enabled = enabled;
+    }
+
+    public Client getClient() {
+        return client;
+    }
+
+    public void setClient(Client client) {
+        this.client = client;
+    }
+
+    public String getMedium() {
+        return medium;
+    }
+
+    public void setMedium(String medium) {
+        this.medium = medium;
+    }
+
+    public String getPrefix() {
+        return prefix;
+    }
+
+    public void setPrefix(String prefix) {
+        this.prefix = prefix;
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- Google config ---");
+        log.info("Enabled: {}", isEnabled());
+        log.info("Client ID: {}", getClient().getId());
+        log.info("Client secret set? {}", StringUtils.isNotBlank(getClient().getSecret()));
+        log.info("3PID medium: {}", getMedium());
+        log.info("MXID prefix: {}", getPrefix());
+    }
+
+}

src/main/java/io/kamax/mxisd/config/InvitationConfig.java

@@ -0,0 +1,76 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import io.kamax.mxisd.util.GsonUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import javax.annotation.PostConstruct;
+
+@Configuration
+@ConfigurationProperties("invite")
+public class InvitationConfig {
+
+    private final Logger log = LoggerFactory.getLogger(InvitationConfig.class);
+
+    public static class Resolution {
+
+        private boolean recursive;
+        private long timer;
+
+        public boolean isRecursive() {
+            return recursive;
+        }
+
+        public void setRecursive(boolean recursive) {
+            this.recursive = recursive;
+        }
+
+        public long getTimer() {
+            return timer;
+        }
+
+        public void setTimer(long timer) {
+            this.timer = timer;
+        }
+
+    }
+
+    private Resolution resolution;
+
+    public Resolution getResolution() {
+        return resolution;
+    }
+
+    public void setResolution(Resolution resolution) {
+        this.resolution = resolution;
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- Invite config ---");
+        log.info("Resolution: {}", GsonUtil.build().toJson(resolution));
+    }
+
+}

src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java

@@ -20,31 +20,147 @@
 
 package io.kamax.mxisd.config.ldap;
 
-import com.google.gson.Gson;
 import io.kamax.matrix.ThreePidMedium;
-import io.kamax.mxisd.backend.ldap.LdapGenericBackend;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.backend.ldap.LdapBackend;
 import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
 
 import javax.annotation.PostConstruct;
-import java.util.ArrayList;
-import java.util.List;
+import java.util.*;
 
-@Configuration
-@ConfigurationProperties(prefix = "ldap")
-public class LdapConfig {
+public abstract class LdapConfig {
 
-    private Logger log = LoggerFactory.getLogger(LdapConfig.class);
-    private static Gson gson = new Gson();
+    public static class UID {
+
+        private String type;
+        private String value;
+
+        public String getType() {
+            return type;
+        }
+
+        public void setType(String type) {
+            this.type = type;
+        }
+
+        public String getValue() {
+            return value;
+        }
+
+        public void setValue(String value) {
+            this.value = value;
+        }
+
+    }
+
+    public static class Attribute {
+
+        private UID uid;
+        private String name;
+        private Map<String, List<String>> threepid = new HashMap<>();
+
+        public UID getUid() {
+            return uid;
+        }
+
+        public void setUid(UID uid) {
+            this.uid = uid;
+        }
+
+        public String getName() {
+            return name;
+        }
+
+        public void setName(String name) {
+            this.name = name;
+        }
+
+        public Map<String, List<String>> getThreepid() {
+            return threepid;
+        }
+
+        public void setThreepid(Map<String, List<String>> threepid) {
+            this.threepid = threepid;
+        }
+
+    }
+
+    public static class Auth {
 
-    private boolean enabled;
         private String filter;
 
+        public String getFilter() {
+            return filter;
+        }
+
+        public void setFilter(String filter) {
+            this.filter = filter;
+        }
+
+    }
+
+    public static class Connection {
+
+        private boolean tls;
+        private String host;
+        private int port;
+        private String bindDn;
+        private String bindPassword;
+        private String baseDn;
+
+        public boolean isTls() {
+            return tls;
+        }
+
+        public void setTls(boolean tls) {
+            this.tls = tls;
+        }
+
+        public String getHost() {
+            return host;
+        }
+
+        public void setHost(String host) {
+            this.host = host;
+        }
+
+        public int getPort() {
+            return port;
+        }
+
+        public void setPort(int port) {
+            this.port = port;
+        }
+
+        public String getBindDn() {
+            return bindDn;
+        }
+
+        public void setBindDn(String bindDn) {
+            this.bindDn = bindDn;
+        }
+
+        public String getBindPassword() {
+            return bindPassword;
+        }
+
+        public void setBindPassword(String bindPassword) {
+            this.bindPassword = bindPassword;
+        }
+
+        public String getBaseDn() {
+            return baseDn;
+        }
+
+        public void setBaseDn(String baseDn) {
+            this.baseDn = baseDn;
+        }
+
+    }
+
     public static class Directory {
 
         public static class Attribute {
@@ -82,12 +198,55 @@ public class LdapConfig {
 
     }
 
-    @Autowired
-    private LdapConnectionConfig conn;
-    private LdapAttributeConfig attribute;
-    private LdapAuthConfig auth;
+    public static class Identity {
+
+        private String filter;
+        private String token;
+        private Map<String, String> medium = new HashMap<>();
+
+        public String getFilter() {
+            return filter;
+        }
+
+        public void setFilter(String filter) {
+            this.filter = filter;
+        }
+
+        public String getToken() {
+            return token;
+        }
+
+        public void setToken(String token) {
+            this.token = token;
+        }
+
+        public Map<String, String> getMedium() {
+            return medium;
+        }
+
+        public Optional<String> getQuery(String key) {
+            return Optional.ofNullable(medium.get(key));
+        }
+
+        public void setMedium(Map<String, String> medium) {
+            this.medium = medium;
+        }
+
+    }
+
+
+    private Logger log = LoggerFactory.getLogger(LdapConfig.class);
+
+    private boolean enabled;
+    private String filter;
+
+    private Connection connection;
+    private Attribute attribute;
+    private Auth auth;
     private Directory directory;
-    private LdapIdentityConfig identity;
+    private Identity identity;
+
+    protected abstract String getConfigName();
 
     public boolean isEnabled() {
         return enabled;
@@ -105,27 +264,27 @@ public class LdapConfig {
         this.filter = filter;
     }
 
-    public LdapConnectionConfig getConn() {
-        return conn;
+    public Connection getConnection() {
+        return connection;
     }
 
-    public void setConn(LdapConnectionConfig conn) {
-        this.conn = conn;
+    public void setConnection(Connection conn) {
+        this.connection = conn;
     }
 
-    public LdapAttributeConfig getAttribute() {
+    public Attribute getAttribute() {
         return attribute;
     }
 
-    public void setAttribute(LdapAttributeConfig attribute) {
+    public void setAttribute(Attribute attribute) {
         this.attribute = attribute;
     }
 
-    public LdapAuthConfig getAuth() {
+    public Auth getAuth() {
         return auth;
     }
 
-    public void setAuth(LdapAuthConfig auth) {
+    public void setAuth(Auth auth) {
         this.auth = auth;
     }
 
@@ -137,42 +296,45 @@ public class LdapConfig {
         this.directory = directory;
     }
 
-    public LdapIdentityConfig getIdentity() {
+    public Identity getIdentity() {
         return identity;
     }
 
-    public void setIdentity(LdapIdentityConfig identity) {
+    public void setIdentity(Identity identity) {
         this.identity = identity;
     }
 
     @PostConstruct
     public void build() {
-        log.info("--- LDAP Config ---");
+        log.info("--- " + getConfigName() + " Config ---");
         log.info("Enabled: {}", isEnabled());
 
         if (!isEnabled()) {
             return;
         }
 
-        if (StringUtils.isBlank(conn.getHost())) {
+        if (StringUtils.isBlank(connection.getHost())) {
             throw new IllegalStateException("LDAP Host must be configured!");
         }
 
-        if (conn.getPort() < 1 || conn.getPort() > 65535) {
+        if (connection.getPort() < 1 || connection.getPort() > 65535) {
             throw new IllegalStateException("LDAP port is not valid");
         }
 
+        if (StringUtils.isBlank(connection.getBaseDn())) {
+            throw new ConfigurationException("ldap.connection.baseDn");
+        }
+
         if (StringUtils.isBlank(attribute.getUid().getType())) {
             throw new IllegalStateException("Attribute UID Type cannot be empty");
         }
 
-
         if (StringUtils.isBlank(attribute.getUid().getValue())) {
             throw new IllegalStateException("Attribute UID value cannot be empty");
         }
 
         String uidType = attribute.getUid().getType();
-        if (!StringUtils.equals(LdapGenericBackend.UID, uidType) && !StringUtils.equals(LdapGenericBackend.MATRIX_ID, uidType)) {
+        if (!StringUtils.equals(LdapBackend.UID, uidType) && !StringUtils.equals(LdapBackend.MATRIX_ID, uidType)) {
             throw new IllegalArgumentException("Unsupported LDAP UID type: " + uidType);
         }
 
@@ -184,9 +346,9 @@ public class LdapConfig {
         attribute.getThreepid().forEach((k, v) -> {
             if (StringUtils.isBlank(identity.getMedium().get(k))) {
                 if (ThreePidMedium.PhoneNumber.is(k)) {
-                    identity.getMedium().put(k, LdapGenericBackend.buildOrQuery("+" + getIdentity().getToken(), v));
+                    identity.getMedium().put(k, LdapBackend.buildOrQuery("+" + getIdentity().getToken(), v));
                 } else {
-                    identity.getMedium().put(k, LdapGenericBackend.buildOrQuery(getIdentity().getToken(), v));
+                    identity.getMedium().put(k, LdapBackend.buildOrQuery(getIdentity().getToken(), v));
                 }
             }
         });
@@ -195,15 +357,15 @@ public class LdapConfig {
         getDirectory().setFilter(StringUtils.defaultIfBlank(getDirectory().getFilter(), getFilter()));
         getIdentity().setFilter(StringUtils.defaultIfBlank(getIdentity().getFilter(), getFilter()));
 
-        log.info("Host: {}", conn.getHost());
-        log.info("Port: {}", conn.getPort());
-        log.info("Bind DN: {}", conn.getBindDn());
-        log.info("Base DN: {}", conn.getBaseDn());
+        log.info("Host: {}", connection.getHost());
+        log.info("Port: {}", connection.getPort());
+        log.info("Bind DN: {}", connection.getBindDn());
+        log.info("Base DN: {}", connection.getBaseDn());
 
-        log.info("Attribute: {}", gson.toJson(attribute));
-        log.info("Auth: {}", gson.toJson(auth));
-        log.info("Directory: {}", gson.toJson(directory));
-        log.info("Identity: {}", gson.toJson(identity));
+        log.info("Attribute: {}", GsonUtil.get().toJson(attribute));
+        log.info("Auth: {}", GsonUtil.get().toJson(auth));
+        log.info("Directory: {}", GsonUtil.get().toJson(directory));
+        log.info("Identity: {}", GsonUtil.get().toJson(identity));
     }
 
 }

src/main/java/io/kamax/mxisd/config/ldap/LdapConnectionConfig.java

@@ -1,85 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.ldap;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-@Configuration
-@ConfigurationProperties(prefix = "ldap.connection")
-public class LdapConnectionConfig {
-
-    private boolean tls;
-    private String host;
-    private int port;
-    private String bindDn;
-    private String bindPassword;
-    private String baseDn;
-
-    public boolean isTls() {
-        return tls;
-    }
-
-    public void setTls(boolean tls) {
-        this.tls = tls;
-    }
-
-    public String getHost() {
-        return host;
-    }
-
-    public void setHost(String host) {
-        this.host = host;
-    }
-
-    public int getPort() {
-        return port;
-    }
-
-    public void setPort(int port) {
-        this.port = port;
-    }
-
-    public String getBindDn() {
-        return bindDn;
-    }
-
-    public void setBindDn(String bindDn) {
-        this.bindDn = bindDn;
-    }
-
-    public String getBindPassword() {
-        return bindPassword;
-    }
-
-    public void setBindPassword(String bindPassword) {
-        this.bindPassword = bindPassword;
-    }
-
-    public String getBaseDn() {
-        return baseDn;
-    }
-
-    public void setBaseDn(String baseDn) {
-        this.baseDn = baseDn;
-    }
-
-}

src/main/java/io/kamax/mxisd/config/ldap/LdapIdentityConfig.java

@@ -1,66 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.ldap;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-import java.util.HashMap;
-import java.util.Map;
-import java.util.Optional;
-
-@Configuration
-@ConfigurationProperties(prefix = "ldap.identity")
-public class LdapIdentityConfig {
-
-    private String filter;
-    private String token;
-    private Map<String, String> medium = new HashMap<>();
-
-    public String getFilter() {
-        return filter;
-    }
-
-    public void setFilter(String filter) {
-        this.filter = filter;
-    }
-
-    public String getToken() {
-        return token;
-    }
-
-    public void setToken(String token) {
-        this.token = token;
-    }
-
-    public Map<String, String> getMedium() {
-        return medium;
-    }
-
-    public Optional<String> getQuery(String key) {
-        return Optional.ofNullable(medium.get(key));
-    }
-
-    public void setMedium(Map<String, String> medium) {
-        this.medium = medium;
-    }
-
-}

src/main/java/io/kamax/mxisd/config/ldap/generic/GenericLdapConfig.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sàrl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,32 +18,21 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.config.ldap;
+package io.kamax.mxisd.config.ldap.generic;
 
+import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
+import org.springframework.context.annotation.Primary;
 
 @Configuration
-@ConfigurationProperties(prefix = "ldap.attribute.uid")
-public class LdapAttributeUidConfig {
+@ConfigurationProperties(prefix = "ldap")
+@Primary
+public class GenericLdapConfig extends LdapConfig {
 
-    private String type;
-    private String value;
-
-    public String getType() {
-        return type;
-    }
-
-    public void setType(String type) {
-        this.type = type;
-    }
-
-    public String getValue() {
-        return value;
-    }
-
-    public void setValue(String value) {
-        this.value = value;
+    @Override
+    protected String getConfigName() {
+        return "Generic LDAP";
     }
 
 }

src/main/java/io/kamax/mxisd/config/ldap/netiq/NetIqLdapConfig.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sàrl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,23 +18,19 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.config.ldap;
+package io.kamax.mxisd.config.ldap.netiq;
 
+import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
 @Configuration
-@ConfigurationProperties(prefix = "ldap.auth")
-public class LdapAuthConfig {
+@ConfigurationProperties(prefix = "netiq")
+public class NetIqLdapConfig extends LdapConfig {
 
-    private String filter;
-
-    public String getFilter() {
-        return filter;
-    }
-
-    public void setFilter(String filter) {
-        this.filter = filter;
+    @Override
+    protected String getConfigName() {
+        return "NetIQ eDirectory";
     }
 
 }

src/main/java/io/kamax/mxisd/config/memory/MemoryIdentityConfig.java

@@ -0,0 +1,68 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config.memory;
+
+import org.springframework.stereotype.Component;
+
+import java.util.ArrayList;
+import java.util.List;
+
+@Component
+public class MemoryIdentityConfig {
+
+    private String username;
+    private String password;
+    private List<MemoryThreePid> threepids = new ArrayList<>();
+    private List<String> roles = new ArrayList<>();
+
+    public String getUsername() {
+        return username;
+    }
+
+    public void setUsername(String username) {
+        this.username = username;
+    }
+
+    public String getPassword() {
+        return password;
+    }
+
+    public void setPassword(String password) {
+        this.password = password;
+    }
+
+    public List<MemoryThreePid> getThreepids() {
+        return threepids;
+    }
+
+    public void setThreepids(List<MemoryThreePid> threepids) {
+        this.threepids = threepids;
+    }
+
+    public List<String> getRoles() {
+        return roles;
+    }
+
+    public void setRoles(List<String> roles) {
+        this.roles = roles;
+    }
+
+}

src/main/java/io/kamax/mxisd/config/memory/MemoryStoreConfig.java

@@ -0,0 +1,51 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config.memory;
+
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import java.util.List;
+
+@Configuration
+@ConfigurationProperties("memory")
+public class MemoryStoreConfig {
+
+    private boolean enabled;
+    private List<MemoryIdentityConfig> identities;
+
+    public boolean isEnabled() {
+        return enabled;
+    }
+
+    public void setEnabled(boolean enabled) {
+        this.enabled = enabled;
+    }
+
+    public List<MemoryIdentityConfig> getIdentities() {
+        return identities;
+    }
+
+    public void setIdentities(List<MemoryIdentityConfig> identities) {
+        this.identities = identities;
+    }
+
+}

src/main/java/io/kamax/mxisd/config/memory/MemoryThreePid.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Maxime Dor
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,52 +18,33 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd;
+package io.kamax.mxisd.config.memory;
 
-// FIXME this should be in matrix-java-sdk
-public class ThreePid {
+import io.kamax.matrix._ThreePid;
+import org.springframework.stereotype.Component;
+
+@Component
+public class MemoryThreePid implements _ThreePid {
 
     private String medium;
     private String address;
 
-    public ThreePid(ThreePid tpid) {
-        this(tpid.getMedium(), tpid.getAddress());
-    }
-
-    public ThreePid(String medium, String address) {
-        this.medium = medium;
-        this.address = address;
-    }
-
+    @Override
     public String getMedium() {
         return medium;
     }
 
+    public void setMedium(String medium) {
+        this.medium = medium;
+    }
+
+    @Override
     public String getAddress() {
         return address;
     }
 
-    @Override
-    public String toString() {
-        return getMedium() + ":" + getAddress();
-    }
-
-    @Override
-    public boolean equals(Object o) {
-        if (this == o) return true;
-        if (o == null || getClass() != o.getClass()) return false;
-
-        ThreePid threePid = (ThreePid) o;
-
-        if (!medium.equals(threePid.medium)) return false;
-        return address.equals(threePid.address);
-    }
-
-    @Override
-    public int hashCode() {
-        int result = medium.hashCode();
-        result = 31 * result + address.hashCode();
-        return result;
+    public void setAddress(String address) {
+        this.address = address;
     }
 
 }

src/main/java/io/kamax/mxisd/config/sql/GenericSqlProviderConfig.java

@@ -24,21 +24,14 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 import org.springframework.context.annotation.Primary;
 
-import javax.annotation.PostConstruct;
-
 @Configuration
 @ConfigurationProperties("sql")
 @Primary
-public class SqlProviderConfig extends SqlConfig {
+public class GenericSqlProviderConfig extends SqlConfig {
 
     @Override
     protected String getProviderName() {
         return "Generic SQL";
     }
 
-    @PostConstruct
-    public void build() {
-        super.build();
-    }
-
 }

src/main/java/io/kamax/mxisd/config/sql/SqlConfig.java

@@ -4,6 +4,7 @@ import io.kamax.mxisd.util.GsonUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.annotation.PostConstruct;
 import java.util.HashMap;
 import java.util.Map;
 
@@ -36,22 +37,22 @@ public abstract class SqlConfig {
 
     public static class Type {
 
-        private SqlProviderConfig.Query name = new SqlProviderConfig.Query();
-        private SqlProviderConfig.Query threepid = new SqlProviderConfig.Query();
+        private Query name = new Query();
+        private Query threepid = new Query();
 
-        public SqlProviderConfig.Query getName() {
+        public Query getName() {
             return name;
         }
 
-        public void setName(SqlProviderConfig.Query name) {
+        public void setName(Query name) {
             this.name = name;
         }
 
-        public SqlProviderConfig.Query getThreepid() {
+        public Query getThreepid() {
             return threepid;
         }
 
-        public void setThreepid(SqlProviderConfig.Query threepid) {
+        public void setThreepid(Query threepid) {
             this.threepid = threepid;
         }
 
@@ -74,7 +75,7 @@ public abstract class SqlConfig {
     public static class Directory {
 
         private Boolean enabled;
-        private SqlProviderConfig.Type query = new SqlProviderConfig.Type();
+        private Type query = new Type();
 
         public Boolean isEnabled() {
             return enabled;
@@ -84,11 +85,11 @@ public abstract class SqlConfig {
             this.enabled = enabled;
         }
 
-        public SqlProviderConfig.Type getQuery() {
+        public Type getQuery() {
             return query;
         }
 
-        public void setQuery(SqlProviderConfig.Type query) {
+        public void setQuery(Type query) {
             this.query = query;
         }
 
@@ -135,12 +136,41 @@ public abstract class SqlConfig {
 
     }
 
+    public static class ProfileThreepids {
+
+        private String query;
+
+        public String getQuery() {
+            return query;
+        }
+
+        public void setQuery(String query) {
+            this.query = query;
+        }
+
+    }
+
+    public static class Profile {
+
+        private ProfileThreepids threepid = new ProfileThreepids();
+
+        public ProfileThreepids getThreepid() {
+            return threepid;
+        }
+
+        public void setThreepid(ProfileThreepids threepid) {
+            this.threepid = threepid;
+        }
+
+    }
+
     private boolean enabled;
     private String type;
     private String connection;
-    private SqlProviderConfig.Auth auth = new SqlProviderConfig.Auth();
-    private SqlProviderConfig.Directory directory = new SqlProviderConfig.Directory();
-    private SqlProviderConfig.Identity identity = new SqlProviderConfig.Identity();
+    private Auth auth = new Auth();
+    private Directory directory = new Directory();
+    private Identity identity = new Identity();
+    private Profile profile = new Profile();
 
     public boolean isEnabled() {
         return enabled;
@@ -166,35 +196,41 @@ public abstract class SqlConfig {
         this.connection = connection;
     }
 
-    public SqlProviderConfig.Auth getAuth() {
+    public Auth getAuth() {
         return auth;
     }
 
-    public void setAuth(SqlProviderConfig.Auth auth) {
+    public void setAuth(Auth auth) {
         this.auth = auth;
     }
 
-    public SqlProviderConfig.Directory getDirectory() {
+    public Directory getDirectory() {
         return directory;
     }
 
-    public void setDirectory(SqlProviderConfig.Directory directory) {
+    public void setDirectory(Directory directory) {
         this.directory = directory;
     }
 
-    public SqlProviderConfig.Identity getIdentity() {
+    public Identity getIdentity() {
         return identity;
     }
 
-    public void setIdentity(SqlProviderConfig.Identity identity) {
+    public void setIdentity(Identity identity) {
         this.identity = identity;
     }
 
+    public Profile getProfile() {
+        return profile;
+    }
+
+    public void setProfile(Profile profile) {
+        this.profile = profile;
+    }
+
     protected abstract String getProviderName();
 
-    public void build() {
-        log.info("--- " + getProviderName() + " Provider config ---");
-
+    protected void doBuild() {
         if (getAuth().isEnabled() == null) {
             getAuth().setEnabled(isEnabled());
         }
@@ -206,6 +242,13 @@ public abstract class SqlConfig {
         if (getIdentity().isEnabled() == null) {
             getIdentity().setEnabled(isEnabled());
         }
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- " + getProviderName() + " Provider config ---");
+
+        doBuild();
 
         log.info("Enabled: {}", isEnabled());
         if (isEnabled()) {
@@ -214,7 +257,9 @@ public abstract class SqlConfig {
             log.info("Auth enabled: {}", getAuth().isEnabled());
             log.info("Directory queries: {}", GsonUtil.build().toJson(getDirectory().getQuery()));
             log.info("Identity type: {}", getIdentity().getType());
+            log.info("3PID mapping query: {}", getIdentity().getQuery());
             log.info("Identity medium queries: {}", GsonUtil.build().toJson(getIdentity().getMedium()));
+            log.info("Profile 3PID query: {}", getProfile().getThreepid().getQuery());
         }
     }
 

src/main/java/io/kamax/mxisd/config/sql/synapse/SynapseSqlProviderConfig.java

@@ -21,6 +21,7 @@
 package io.kamax.mxisd.config.sql.synapse;
 
 import io.kamax.mxisd.config.sql.SqlConfig;
+import org.apache.commons.lang.StringUtils;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
@@ -36,8 +37,23 @@ public class SynapseSqlProviderConfig extends SqlConfig {
     }
 
     @PostConstruct
-    public void build() {
-        super.build();
+    public void doBuild() {
+        super.doBuild();
+        // FIXME check that the DB is not the mxisd one
+        // See https://matrix.to/#/!NPRUEisLjcaMtHIzDr:kamax.io/$1509377583327omXkC:kamax.io
+
+        getAuth().setEnabled(false); // Synapse does the auth, we only act as a directory/identity service.
+
+        if (getDirectory().isEnabled()) {
+            //FIXME set default queries for name and threepid
+        }
+
+        if (getIdentity().isEnabled()) {
+            if (StringUtils.isBlank(getIdentity().getType())) {
+                getIdentity().setType("mxid");
+                getIdentity().setQuery("SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?");
+            }
+        }
     }
 
 }

src/main/java/io/kamax/mxisd/config/wordpress/WordpressConfig.java

@@ -0,0 +1,175 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config.wordpress;
+
+import io.kamax.mxisd.exception.ConfigurationException;
+import org.apache.commons.lang.StringUtils;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import javax.annotation.PostConstruct;
+import java.util.Map;
+
+@Configuration
+@ConfigurationProperties("wordpress")
+public class WordpressConfig {
+
+    public static class Credential {
+
+        private String username;
+        private String password;
+
+        public String getUsername() {
+            return username;
+        }
+
+        public void setUsername(String username) {
+            this.username = username;
+        }
+
+        public String getPassword() {
+            return password;
+        }
+
+        public void setPassword(String password) {
+            this.password = password;
+        }
+
+    }
+
+    public static class Rest {
+
+        private Credential credential = new Credential();
+        private String base;
+
+        public String getBase() {
+            return base;
+        }
+
+        public void setBase(String base) {
+            this.base = base;
+        }
+
+        public Credential getCredential() {
+            return credential;
+        }
+
+        public void setCredential(Credential credential) {
+            this.credential = credential;
+        }
+
+    }
+
+    public static class Query {
+
+        private Map<String, String> threepid;
+        private Map<String, String> directory;
+
+        public Map<String, String> getThreepid() {
+            return threepid;
+        }
+
+        public void setThreepid(Map<String, String> threepid) {
+            this.threepid = threepid;
+        }
+
+        public Map<String, String> getDirectory() {
+            return directory;
+        }
+
+        public void setDirectory(Map<String, String> directory) {
+            this.directory = directory;
+        }
+
+    }
+
+    public static class Sql {
+
+        private String type;
+        private String connection;
+        private Query query;
+
+        public String getType() {
+            return type;
+        }
+
+        public void setType(String type) {
+            this.type = type;
+        }
+
+        public String getConnection() {
+            return connection;
+        }
+
+        public void setConnection(String connection) {
+            this.connection = connection;
+        }
+
+        public Query getQuery() {
+            return query;
+        }
+
+        public void setQuery(Query query) {
+            this.query = query;
+        }
+
+    }
+
+    private boolean enabled;
+    private Rest rest = new Rest();
+    private Sql sql = new Sql();
+
+    public boolean isEnabled() {
+        return enabled;
+    }
+
+    public void setEnabled(boolean enabled) {
+        this.enabled = enabled;
+    }
+
+    public Rest getRest() {
+        return rest;
+    }
+
+    public void setRest(Rest rest) {
+        this.rest = rest;
+    }
+
+    public Sql getSql() {
+        return sql;
+    }
+
+    public void setSql(Sql sql) {
+        this.sql = sql;
+    }
+
+    @PostConstruct
+    public void build() {
+        if (!isEnabled()) {
+            return;
+        }
+
+        if (StringUtils.isBlank(getRest().getBase())) {
+            throw new ConfigurationException("wordpress.rest.base");
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/controller/DefaultExceptionHandler.java

@@ -54,21 +54,40 @@ public class DefaultExceptionHandler {
         return gson.toJson(obj);
     }
 
+    @ExceptionHandler(RemoteLoginException.class)
+    public String handle(HttpServletRequest request, HttpServletResponse response, RemoteLoginException e) {
+        if (e.getErrorBodyMsgResp() != null) {
+            response.setStatus(e.getStatus());
+            log.info("Request {} {} - Error {}: {}", request.getMethod(), request.getRequestURL(), e.getErrorCode(), e.getError());
+            return gson.toJson(e.getErrorBodyMsgResp());
+        }
+        return handleGeneric(request, response, e);
+    }
+
     @ExceptionHandler(InternalServerError.class)
-    public String handle(HttpServletRequest req, InternalServerError e, HttpServletResponse response) {
+    public String handle(HttpServletRequest request, HttpServletResponse response, InternalServerError e) {
         if (StringUtils.isNotBlank(e.getInternalReason())) {
             log.error("Reference #{} - {}", e.getReference(), e.getInternalReason());
         } else {
             log.error("Reference #{}", e);
         }
 
-        return handleGeneric(req, e, response);
+        return handleGeneric(request, response, e);
     }
 
-    @ExceptionHandler(MatrixException.class)
-    public String handleGeneric(HttpServletRequest req, MatrixException e, HttpServletResponse response) {
+    @ExceptionHandler(FeatureNotAvailable.class)
+    public String handle(HttpServletRequest request, HttpServletResponse response, FeatureNotAvailable e) {
+        if (StringUtils.isNotBlank(e.getInternalReason())) {
+            log.error("Feature not available: {}", e.getInternalReason());
+        }
+
+        return handleGeneric(request, response, e);
+    }
+
+    @ExceptionHandler(HttpMatrixException.class)
+    public String handleGeneric(HttpServletRequest request, HttpServletResponse response, HttpMatrixException e) {
         response.setStatus(e.getStatus());
-        return handle(req, e.getErrorCode(), e.getError());
+        return handle(request, e.getErrorCode(), e.getError());
     }
 
     @ResponseStatus(HttpStatus.BAD_REQUEST)

src/main/java/io/kamax/mxisd/controller/PingController.java

@@ -0,0 +1,38 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.controller;
+
+import org.springframework.http.MediaType;
+import org.springframework.web.bind.annotation.CrossOrigin;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RestController;
+
+@RestController
+@CrossOrigin
+@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
+public class PingController {
+
+    @RequestMapping(value = "/_matrix/identity/api/v1")
+    public String ping() {
+        return "{}";
+    }
+
+}

src/main/java/io/kamax/mxisd/controller/ProxyController.java

@@ -0,0 +1,54 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.controller;
+
+import io.kamax.mxisd.exception.AccessTokenNotFoundException;
+import io.kamax.mxisd.util.OptionalUtil;
+import org.thymeleaf.util.StringUtils;
+
+import javax.servlet.http.HttpServletRequest;
+import java.util.Optional;
+
+public class ProxyController {
+
+    private final static String headerName = "Authorization";
+    private final static String headerValuePrefix = "Bearer ";
+    private final static String parameterName = "access_token";
+
+    Optional<String> findAccessTokenInHeaders(HttpServletRequest request) {
+        return Optional.ofNullable(request.getHeader(headerName))
+                .filter(header -> StringUtils.startsWith(header, headerValuePrefix))
+                .map(header -> header.substring(headerValuePrefix.length()));
+    }
+
+    Optional<String> findAccessTokenInQuery(HttpServletRequest request) {
+        return Optional.ofNullable(request.getParameter(parameterName));
+    }
+
+    public Optional<String> findAccessToken(HttpServletRequest request) {
+        return OptionalUtil.findFirst(() -> findAccessTokenInHeaders(request), () -> findAccessTokenInQuery(request));
+    }
+
+    public String getAccessToken(HttpServletRequest request) {
+        return findAccessToken(request).orElseThrow(AccessTokenNotFoundException::new);
+    }
+
+}

src/main/java/io/kamax/mxisd/controller/auth/v1/AuthController.java

@@ -20,15 +20,27 @@
 
 package io.kamax.mxisd.controller.auth.v1;
 
-import com.google.gson.Gson;
-import com.google.gson.JsonElement;
-import com.google.gson.JsonObject;
+import com.google.gson.*;
+import com.google.i18n.phonenumbers.NumberParseException;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import com.google.i18n.phonenumbers.Phonenumber;
 import io.kamax.mxisd.auth.AuthManager;
 import io.kamax.mxisd.auth.UserAuthResult;
 import io.kamax.mxisd.controller.auth.v1.io.CredentialsValidationResponse;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
 import io.kamax.mxisd.exception.JsonMemberNotFoundException;
+import io.kamax.mxisd.exception.RemoteLoginException;
+import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.util.GsonParser;
 import io.kamax.mxisd.util.GsonUtil;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -39,13 +51,18 @@ import org.springframework.web.bind.annotation.RequestMethod;
 import org.springframework.web.bind.annotation.RestController;
 
 import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
 import java.io.IOException;
+import java.net.URI;
 
 @RestController
 @CrossOrigin
 @RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
 public class AuthController {
 
+    // TODO export into SDK
+    private static final String logV1Url = "/_matrix/client/r0/login";
+
     private Logger log = LoggerFactory.getLogger(AuthController.class);
 
     private Gson gson = GsonUtil.build();
@@ -54,6 +71,23 @@ public class AuthController {
     @Autowired
     private AuthManager mgr;
 
+    @Autowired
+    private LookupStrategy strategy;
+
+    @Autowired
+    private ClientDnsOverwrite dns;
+
+    @Autowired
+    private CloseableHttpClient client;
+
+    private String resolveProxyUrl(HttpServletRequest req) {
+        URI target = URI.create(req.getRequestURL().toString());
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
+
     @RequestMapping(value = "/_matrix-internal/identity/v1/check_credentials", method = RequestMethod.POST)
     public String checkCredentials(HttpServletRequest req) {
         try {
@@ -84,4 +118,112 @@ public class AuthController {
         }
     }
 
+    @RequestMapping(value = logV1Url, method = RequestMethod.GET)
+    public String getLogin(HttpServletRequest req, HttpServletResponse res) {
+        try (CloseableHttpResponse hsResponse = client.execute(new HttpGet(resolveProxyUrl(req)))) {
+            res.setStatus(hsResponse.getStatusLine().getStatusCode());
+            return EntityUtils.toString(hsResponse.getEntity());
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    @RequestMapping(value = logV1Url, method = RequestMethod.POST)
+    public String login(HttpServletRequest req) {
+        try {
+            JsonObject reqJsonObject = parser.parse(req.getInputStream());
+
+            // find 3PID in main object
+            GsonUtil.findPrimitive(reqJsonObject, "medium").ifPresent(medium -> {
+                GsonUtil.findPrimitive(reqJsonObject, "address").ifPresent(address -> {
+                    log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
+                    strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
+                        reqJsonObject.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
+                        reqJsonObject.remove("medium");
+                        reqJsonObject.remove("address");
+                    });
+                });
+            });
+
+            // find 3PID in 'identifier' object
+            GsonUtil.findObj(reqJsonObject, "identifier").ifPresent(identifier -> {
+                GsonUtil.findPrimitive(identifier, "type").ifPresent(type -> {
+
+                    if (StringUtils.equals(type.getAsString(), "m.id.thirdparty")) {
+                        GsonUtil.findPrimitive(identifier, "medium").ifPresent(medium -> {
+                            GsonUtil.findPrimitive(identifier, "address").ifPresent(address -> {
+                                log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
+                                strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
+                                    identifier.addProperty("type", "m.id.user");
+                                    identifier.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
+                                    identifier.remove("medium");
+                                    identifier.remove("address");
+                                });
+                            });
+                        });
+                    }
+
+                    if (StringUtils.equals(type.getAsString(), "m.id.phone")) {
+                        GsonUtil.findPrimitive(identifier, "number").ifPresent(number -> {
+                            GsonUtil.findPrimitive(identifier, "country").ifPresent(country -> {
+                                log.info("Login request with phone '{}'-'{}'", country.getAsString(), number.getAsString());
+                                try {
+                                    PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+                                    Phonenumber.PhoneNumber phoneNumber = phoneUtil.parse(number.getAsString(), country.getAsString());
+                                    String canon_phoneNumber = phoneUtil.format(phoneNumber, PhoneNumberUtil.PhoneNumberFormat.E164).replace("+", "");
+                                    String medium = "msisdn";
+                                    strategy.findLocal(medium, canon_phoneNumber).ifPresent(lookupDataOpt -> {
+                                        identifier.addProperty("type", "m.id.user");
+                                        identifier.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
+                                        identifier.remove("country");
+                                        identifier.remove("number");
+                                    });
+                                } catch (NumberParseException e) {
+                                    throw new RuntimeException(e);
+                                }
+                            });
+                        });
+                    }
+                });
+            });
+
+            // invoke 'login' on homeserver
+            HttpPost httpPost = RestClientUtils.post(resolveProxyUrl(req), gson, reqJsonObject);
+            try (CloseableHttpResponse httpResponse = client.execute(httpPost)) {
+                // check http status
+                int status = httpResponse.getStatusLine().getStatusCode();
+                log.info("http status = {}", status);
+                if (status != 200) {
+                    // try to get possible json error message from response
+                    // otherwise just get returned plain error message
+                    String errcode = String.valueOf(httpResponse.getStatusLine().getStatusCode());
+                    String error = EntityUtils.toString(httpResponse.getEntity());
+                    if (httpResponse.getEntity() != null) {
+                        try {
+                            JsonObject bodyJson = new JsonParser().parse(error).getAsJsonObject();
+                            if (bodyJson.has("errcode")) {
+                                errcode = bodyJson.get("errcode").getAsString();
+                            }
+                            if (bodyJson.has("error")) {
+                                error = bodyJson.get("error").getAsString();
+                            }
+                            throw new RemoteLoginException(status, errcode, error, bodyJson);
+                        } catch (JsonSyntaxException e) {
+                            log.warn("Response body is not JSON");
+                        }
+                    }
+                    throw new RemoteLoginException(status, errcode, error);
+                }
+
+                /// return response
+                JsonObject respJsonObject = parser.parseOptional(httpResponse).get();
+                return gson.toJson(respJsonObject);
+            } catch (IOException e) {
+                throw new RuntimeException(e);
+            }
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
 }

src/main/java/io/kamax/mxisd/controller/auth/v1/RegistrationController.java

@@ -0,0 +1,159 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.controller.auth.v1;
+
+import com.google.api.client.googleapis.auth.oauth2.GoogleIdToken;
+import com.google.gson.Gson;
+import com.google.gson.JsonObject;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.backend.google.GoogleProviderBackend;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.profile.ProfileManager;
+import io.kamax.mxisd.util.GsonParser;
+import io.kamax.mxisd.util.GsonUtil;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.MediaType;
+import org.springframework.web.bind.annotation.CrossOrigin;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestMethod;
+import org.springframework.web.bind.annotation.RestController;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import java.io.IOException;
+import java.net.URI;
+import java.security.GeneralSecurityException;
+import java.util.ArrayList;
+import java.util.List;
+
+@RestController
+@CrossOrigin
+@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
+public class RegistrationController {
+
+    private final Logger log = LoggerFactory.getLogger(RegistrationController.class);
+
+    private final String registerV1Url = "/_matrix/client/r0/register";
+
+    private GoogleProviderBackend google;
+    private ProfileManager pMgr;
+    private ClientDnsOverwrite dns;
+    private CloseableHttpClient client;
+    private Gson gson;
+    private GsonParser parser;
+
+    @Autowired
+    public RegistrationController(GoogleProviderBackend google, ProfileManager pMgr, ClientDnsOverwrite dns, CloseableHttpClient client) {
+        this.google = google;
+        this.pMgr = pMgr;
+        this.dns = dns;
+        this.client = client;
+        this.gson = GsonUtil.build();
+        this.parser = new GsonParser(gson);
+    }
+
+    private String resolveProxyUrl(HttpServletRequest req) {
+        URI target = URI.create(req.getRequestURL().toString());
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
+
+    @RequestMapping(path = registerV1Url, method = RequestMethod.GET)
+    public String getLogin(HttpServletRequest req, HttpServletResponse res) {
+        try (CloseableHttpResponse hsResponse = client.execute(new HttpGet(resolveProxyUrl(req)))) {
+            res.setStatus(hsResponse.getStatusLine().getStatusCode());
+            return EntityUtils.toString(hsResponse.getEntity());
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    @RequestMapping(path = registerV1Url, method = RequestMethod.POST)
+    public String register(HttpServletRequest req, HttpServletResponse res) {
+        List<ThreePid> ids = new ArrayList<>();
+        try {
+            JsonObject reqJsonObject = parser.parse(req.getInputStream());
+            GsonUtil.findObj(reqJsonObject, "auth").ifPresent(auth -> {
+                GsonUtil.findPrimitive(auth, "type").ifPresent(type -> {
+                    if (StringUtils.equals("io.kamax.google.auth", type.getAsString())) {
+                        log.info("Got registration attempt with Google account");
+                        if (!auth.has("googleId")) {
+                            throw new IllegalArgumentException("Google ID is missing");
+                        }
+
+                        String gId = auth.get("googleId").getAsString();
+                        try {
+                            GoogleIdToken token = google.extractToken(reqJsonObject.get("password").getAsString()).orElseThrow(() -> new IllegalArgumentException("Google ID Token is missing or invalid"));
+                            if (!StringUtils.equals(gId, token.getPayload().getSubject())) {
+                                throw new IllegalArgumentException("Google ID does not match token");
+                            }
+                            log.info("Google ID: {}", gId);
+
+                            ids.addAll(google.extractThreepids(token));
+
+                            auth.addProperty("type", "m.login.dummy");
+                            auth.remove("googleId");
+                            reqJsonObject.addProperty("username", "g-" + gId);
+                            reqJsonObject.addProperty("password", "");
+                        } catch (IOException | GeneralSecurityException e) {
+                            throw new RuntimeException(e);
+                        }
+                    }
+                });
+            });
+
+            log.info("Sending body: {}", gson.toJson(reqJsonObject));
+            HttpPost httpPost = RestClientUtils.post(resolveProxyUrl(req), gson, reqJsonObject);
+            try (CloseableHttpResponse httpResponse = client.execute(httpPost)) {
+                int sc = httpResponse.getStatusLine().getStatusCode();
+                String body = EntityUtils.toString(httpResponse.getEntity());
+                JsonObject json = parser.parse(body);
+                if (sc == 200 && json.has("user_id")) {
+                    // Required here as synapse doesn't call pass provider on register
+                    log.info("User was registered, adding 3PIDs");
+                    _MatrixID mxid = new MatrixID(json.get("user_id").getAsString());
+                    for (ThreePid tpid : ids) {
+                        pMgr.addThreepid(mxid, tpid);
+                    }
+                }
+                res.setStatus(sc);
+                return body;
+            }
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/controller/auth/v1/io/CredentialsValidationResponse.java

@@ -20,7 +20,7 @@
 
 package io.kamax.mxisd.controller.auth.v1.io;
 
-import io.kamax.mxisd.ThreePid;
+import io.kamax.matrix.ThreePid;
 
 import java.util.HashSet;
 import java.util.Set;

src/main/java/io/kamax/mxisd/controller/directory/v1/UserDirectoryController.java

@@ -21,6 +21,7 @@
 package io.kamax.mxisd.controller.directory.v1;
 
 import com.google.gson.Gson;
+import io.kamax.mxisd.controller.ProxyController;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchRequest;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.DirectoryManager;
@@ -28,7 +29,10 @@ import io.kamax.mxisd.util.GsonParser;
 import io.kamax.mxisd.util.GsonUtil;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.http.MediaType;
-import org.springframework.web.bind.annotation.*;
+import org.springframework.web.bind.annotation.CrossOrigin;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestMethod;
+import org.springframework.web.bind.annotation.RestController;
 
 import javax.servlet.http.HttpServletRequest;
 import java.io.IOException;
@@ -37,7 +41,7 @@ import java.net.URI;
 @RestController
 @CrossOrigin
 @RequestMapping(path = "/_matrix/client/r0/user_directory", produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
-public class UserDirectoryController {
+public class UserDirectoryController extends ProxyController {
 
     private Gson gson = GsonUtil.build();
     private GsonParser parser = new GsonParser(gson);
@@ -46,7 +50,8 @@ public class UserDirectoryController {
     private DirectoryManager mgr;
 
     @RequestMapping(path = "/search", method = RequestMethod.POST)
-    public String search(HttpServletRequest request, @RequestParam("access_token") String accessToken) throws IOException {
+    public String search(HttpServletRequest request) throws IOException {
+        String accessToken = getAccessToken(request);
         UserDirectorySearchRequest searchQuery = parser.parse(request, UserDirectorySearchRequest.class);
         URI target = URI.create(request.getRequestURL().toString());
         UserDirectorySearchResult result = mgr.search(target, accessToken, searchQuery.getSearchTerm());

src/main/java/io/kamax/mxisd/controller/identity/v1/InvitationController.java

@@ -22,13 +22,13 @@ package io.kamax.mxisd.controller.identity.v1;
 
 import com.google.gson.Gson;
 import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.crypto.KeyManager;
 import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.controller.identity.v1.io.ThreePidInviteReplyIO;
 import io.kamax.mxisd.invitation.IThreePidInvite;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.invitation.InvitationManager;
 import io.kamax.mxisd.invitation.ThreePidInvite;
-import io.kamax.mxisd.key.KeyManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;

src/main/java/io/kamax/mxisd/controller/identity/v1/KeyController.java

@@ -22,9 +22,9 @@ package io.kamax.mxisd.controller.identity.v1;
 
 import com.google.gson.Gson;
 import com.google.gson.JsonObject;
+import io.kamax.matrix.crypto.KeyManager;
 import io.kamax.mxisd.controller.identity.v1.io.KeyValidityJson;
 import io.kamax.mxisd.exception.BadRequestException;
-import io.kamax.mxisd.key.KeyManager;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -64,7 +64,7 @@ public class KeyController {
 
     @RequestMapping(value = "/pubkey/ephemeral/isvalid", method = GET)
     public String checkEphemeralKeyValidity(HttpServletRequest request) {
-        log.warn("Ephemeral key was request but no ephemeral key are generated, replying not valid");
+        log.warn("Ephemeral key was requested but no ephemeral key are generated, replying not valid");
 
         return invalidKey;
     }

src/main/java/io/kamax/mxisd/controller/identity/v1/MappingController.java

@@ -22,11 +22,14 @@ package io.kamax.mxisd.controller.identity.v1;
 
 import com.google.gson.Gson;
 import com.google.gson.JsonObject;
+import io.kamax.matrix.crypto.SignatureManager;
+import io.kamax.matrix.event.EventKey;
+import io.kamax.matrix.json.MatrixJson;
+import io.kamax.mxisd.config.MatrixConfig;
 import io.kamax.mxisd.controller.identity.v1.io.SingeLookupReplyJson;
 import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.lookup.*;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
-import io.kamax.mxisd.signature.SignatureManager;
 import io.kamax.mxisd.util.GsonParser;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
@@ -57,6 +60,9 @@ public class MappingController {
     private Gson gson = new Gson();
     private GsonParser parser = new GsonParser(gson);
 
+    @Autowired
+    private MatrixConfig mxCfg;
+
     @Autowired
     private LookupStrategy strategy;
 
@@ -92,17 +98,13 @@ public class MappingController {
         }
 
         SingleLookupReply lookup = lookupOpt.get();
-        if (lookup.isSigned()) {
-            log.info("Lookup is already signed, sending as-is");
-            return lookup.getBody();
-        } else {
-            log.info("Lookup is not signed, signing");
+
+        // FIXME signing should be done in the business model, not in the controller
         JsonObject obj = gson.toJsonTree(new SingeLookupReplyJson(lookup)).getAsJsonObject();
-            obj.add("signatures", signMgr.signMessageGson(gson.toJson(obj)));
+        obj.add(EventKey.Signatures.get(), signMgr.signMessageGson(MatrixJson.encodeCanonical(obj)));
 
         return gson.toJson(obj);
     }
-    }
 
     @RequestMapping(value = "/bulk_lookup", method = POST)
     String bulkLookup(HttpServletRequest request) {

src/main/java/io/kamax/mxisd/controller/identity/v1/SessionController.java

@@ -35,6 +35,8 @@ import org.springframework.web.bind.annotation.RequestParam;
 
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
+import java.net.MalformedURLException;
+import java.net.URL;
 
 import static org.springframework.web.bind.annotation.RequestMethod.GET;
 
@@ -67,7 +69,13 @@ class SessionController {
         ValidationResult r = mgr.validate(sid, secret, token);
         log.info("Session {} was validated", sid);
         if (r.getNextUrl().isPresent()) {
-            String url = srvCfg.getPublicUrl() + r.getNextUrl().get();
+            String url = r.getNextUrl().get();
+            try {
+                url = new URL(url).toString();
+            } catch (MalformedURLException e) {
+                log.info("Session next URL {} is not a valid one, will prepend public URL {}", url, srvCfg.getPublicUrl());
+                url = srvCfg.getPublicUrl() + r.getNextUrl().get();
+            }
             log.info("Session {} validation: next URL is present, redirecting to {}", sid, url);
             return "redirect:" + url;
         } else {

src/main/java/io/kamax/mxisd/controller/identity/v1/SessionRestController.java

@@ -22,8 +22,8 @@ package io.kamax.mxisd.controller.identity.v1;
 
 import com.google.gson.Gson;
 import com.google.gson.JsonObject;
+import io.kamax.matrix.ThreePid;
 import io.kamax.matrix.ThreePidMedium;
-import io.kamax.mxisd.ThreePid;
 import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.config.ViewConfig;
 import io.kamax.mxisd.controller.identity.v1.io.SessionEmailTokenRequestJson;
@@ -104,12 +104,19 @@ public class SessionRestController {
 
         if (ThreePidMedium.PhoneNumber.is(medium)) {
             SessionPhoneTokenRequestJson req = parser.parse(request, SessionPhoneTokenRequestJson.class);
-            return gson.toJson(new Sid(mgr.create(
+            ThreePid threepid = new ThreePid(req.getMedium(), req.getValue());
+
+            String sessionId = mgr.create(
                     request.getRemoteHost(),
-                    new ThreePid(req.getMedium(), req.getValue()),
+                    threepid,
                     req.getSecret(),
                     req.getAttempt(),
-                    req.getNextLink())));
+                    req.getNextLink());
+
+            JsonObject res = new JsonObject();
+            res.addProperty("sid", sessionId);
+            res.addProperty(threepid.getMedium(), threepid.getAddress());
+            return gson.toJson(res);
         }
 
         JsonObject obj = new JsonObject();

src/main/java/io/kamax/mxisd/controller/identity/v1/io/SingeLookupReplyJson.java

@@ -22,9 +22,6 @@ package io.kamax.mxisd.controller.identity.v1.io;
 
 import io.kamax.mxisd.lookup.SingleLookupReply;
 
-import java.util.HashMap;
-import java.util.Map;
-
 public class SingeLookupReplyJson {
 
     private String address;
@@ -33,7 +30,6 @@ public class SingeLookupReplyJson {
     private long not_after;
     private long not_before;
     private long ts;
-    private Map<String, Map<String, String>> signatures = new HashMap<>();
 
     public SingeLookupReplyJson(SingleLookupReply reply) {
         this.address = reply.getRequest().getThreePid();
@@ -68,8 +64,4 @@ public class SingeLookupReplyJson {
         return ts;
     }
 
-    public boolean isSigned() {
-        return signatures != null && !signatures.isEmpty();
-    }
-
 }

src/main/java/io/kamax/mxisd/controller/profile/v1/ProfileController.java

@@ -0,0 +1,105 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.controller.profile.v1;
+
+import com.google.gson.Gson;
+import com.google.gson.JsonElement;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix._ThreePid;
+import io.kamax.mxisd.controller.ProxyController;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.profile.ProfileManager;
+import io.kamax.mxisd.util.GsonUtil;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.MediaType;
+import org.springframework.web.bind.annotation.CrossOrigin;
+import org.springframework.web.bind.annotation.PathVariable;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RestController;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+import java.io.IOException;
+import java.net.URI;
+import java.util.List;
+import java.util.Objects;
+import java.util.Optional;
+
+@RestController
+@CrossOrigin
+@RequestMapping(path = "/_matrix/client/r0/profile", produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
+public class ProfileController extends ProxyController {
+
+    private final Logger log = LoggerFactory.getLogger(ProfileController.class);
+    private final ProfileManager mgr;
+    private final CloseableHttpClient client;
+    private final ClientDnsOverwrite dns;
+    private final JsonParser parser;
+    private final Gson gson;
+
+    @Autowired
+    public ProfileController(ProfileManager mgr, CloseableHttpClient client, ClientDnsOverwrite dns) {
+        this.mgr = mgr;
+        this.client = client;
+        this.dns = dns;
+        this.parser = new JsonParser();
+        this.gson = GsonUtil.build();
+    }
+
+    // FIXME do properly in the SDK (headers, check access token, etc.)
+    private String resolveProxyUrl(HttpServletRequest req) {
+        URI target = URI.create(req.getRequestURL().toString() + (Objects.isNull(req.getQueryString()) ? "" : "?" + req.getQueryString()));
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
+
+    @RequestMapping("/{userId:.+}")
+    public String getProfile(HttpServletRequest req, HttpServletResponse res, @PathVariable String userId) {
+        Optional<String> accessTokenOpt = findAccessToken(req);
+        HttpGet reqOut = new HttpGet(resolveProxyUrl(req));
+        accessTokenOpt.ifPresent(accessToken -> reqOut.addHeader("Authorization", "Bearer " + accessToken));
+
+        try (CloseableHttpResponse hsResponse = client.execute(reqOut)) {
+            res.setStatus(hsResponse.getStatusLine().getStatusCode());
+            JsonElement el = parser.parse(EntityUtils.toString(hsResponse.getEntity()));
+            List<_ThreePid> list = mgr.getThreepids(MatrixID.asAcceptable(userId));
+            if (!list.isEmpty() && el.isJsonObject()) {
+                JsonObject obj = el.getAsJsonObject();
+                obj.add("threepids", GsonUtil.build().toJsonTree(list));
+            }
+            return gson.toJson(el);
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+}

src/main/java/io/kamax/mxisd/controller/profile/v1/ProfileInternalController.java

@@ -0,0 +1,58 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.controller.profile.v1;
+
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.profile.ProfileManager;
+import org.springframework.http.MediaType;
+import org.springframework.web.bind.annotation.CrossOrigin;
+import org.springframework.web.bind.annotation.PathVariable;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RestController;
+
+import java.io.UnsupportedEncodingException;
+import java.net.URLDecoder;
+import java.nio.charset.StandardCharsets;
+
+import static org.springframework.web.bind.annotation.RequestMethod.GET;
+
+@RestController
+@CrossOrigin
+@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
+public class ProfileInternalController {
+
+    private final ProfileManager mgr;
+
+    public ProfileInternalController(ProfileManager mgr) {
+        this.mgr = mgr;
+    }
+
+    @RequestMapping(method = GET, path = "/_matrix-internal/profile/v1/{userId:.+}")
+    public String getProfile(@PathVariable String userId) throws UnsupportedEncodingException {
+        userId = URLDecoder.decode(userId, StandardCharsets.UTF_8.name());
+        _MatrixID mxId = MatrixID.asAcceptable(userId);
+
+        return GsonUtil.get().toJson(GsonUtil.makeObj("roles", GsonUtil.asArray(mgr.getRoles(mxId))));
+    }
+
+}

src/main/java/io/kamax/mxisd/directory/DirectoryManager.java

@@ -23,20 +23,21 @@ package io.kamax.mxisd.directory;
 import com.google.gson.Gson;
 import com.google.gson.JsonSyntaxException;
 import io.kamax.matrix.MatrixErrorInfo;
+import io.kamax.mxisd.config.DirectoryConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchRequest;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.HttpMatrixException;
 import io.kamax.mxisd.exception.InternalServerError;
-import io.kamax.mxisd.exception.MatrixException;
 import io.kamax.mxisd.util.GsonUtil;
 import io.kamax.mxisd.util.RestClientUtils;
 import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.utils.URIBuilder;
 import org.apache.http.entity.ContentType;
 import org.apache.http.impl.client.CloseableHttpClient;
-import org.apache.http.impl.client.HttpClients;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -53,16 +54,19 @@ public class DirectoryManager {
 
     private Logger log = LoggerFactory.getLogger(DirectoryManager.class);
 
+    private DirectoryConfig cfg;
     private List<IDirectoryProvider> providers;
 
     private ClientDnsOverwrite dns;
-    private CloseableHttpClient client;
     private Gson gson;
 
     @Autowired
-    public DirectoryManager(List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+    private CloseableHttpClient client;
+
+    @Autowired
+    public DirectoryManager(DirectoryConfig cfg, List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+        this.cfg = cfg;
         this.dns = dns;
-        this.client = HttpClients.custom().setUserAgent("mxisd").build(); //FIXME centralize
         this.gson = GsonUtil.build();
         this.providers = providers.stream().filter(IDirectoryProvider::isEnabled).collect(Collectors.toList());
 
@@ -75,6 +79,9 @@ public class DirectoryManager {
         log.info("Original request URL: {}", target);
         UserDirectorySearchResult result = new UserDirectorySearchResult();
 
+        if (cfg.getExclude().getHomeserver()) {
+            log.info("Skipping HS directory data, disabled in config");
+        } else {
             URIBuilder builder = dns.transform(target);
             log.info("Querying HS at {}", builder);
             builder.setParameter("access_token", accessToken);
@@ -88,7 +95,12 @@ public class DirectoryManager {
 
                 if (status != 200) {
                     MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
-                throw new MatrixException(status, info.getErrcode(), info.getError());
+                    if (StringUtils.equals("M_UNRECOGNIZED", info.getErrcode())) { // FIXME no hardcoding, use Enum
+                        log.warn("Homeserver does not support Directory feature, skipping");
+                    } else {
+                        log.error("Homeserver returned an error while performing directory search");
+                        throw new HttpMatrixException(status, info.getErrcode(), info.getError());
+                    }
                 }
 
                 UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
@@ -102,6 +114,7 @@ public class DirectoryManager {
             } catch (IOException e) {
                 throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
             }
+        }
 
         for (IDirectoryProvider provider : providers) {
             log.info("Using Directory provider {}", provider.getClass().getSimpleName());