3PID authentication (#60)

Fix for #49

Dockerfile

@@ -1,11 +1,17 @@
 FROM openjdk:8-jre-alpine
 
+RUN apk update && apk add bash && rm -rf /var/lib/apk/* /var/cache/apk/*
+
 VOLUME /etc/mxisd
 VOLUME /var/mxisd
 EXPOSE 8090
 
-ADD build/libs/mxisd.jar /mxisd.jar
-ADD src/docker/start.sh /start.sh
-
 ENV JAVA_OPTS=""
-CMD [ "/start.sh" ]
+ENV CONF_FILE_PATH="/etc/mxisd/mxisd.yaml"
+ENV SIGN_KEY_PATH="/var/mxisd/sign.key"
+ENV SQLITE_DATABASE_PATH="/var/mxisd/mxisd.db"
+
+CMD [ "/start.sh" ]
+
+ADD src/docker/start.sh /start.sh
+ADD build/libs/mxisd.jar /mxisd.jar

README.md

@@ -5,20 +5,20 @@ mxisd - Federated Matrix Identity Server Daemon
 - [Overview](#overview)
 - [Features](#features)
 - [Why use mxisd](#why-use-mxisd)
-- [Quick start](#quick-start)
+- [Getting Started](#getting-started)
 - [Support](#support)
 - [Contribute](#contribute)
 - [FAQ](#faq)
 - [Contact](#contact)
 
 # Overview
-mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with enhanced features.
+mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with [enhanced features](#features).
   
 It is specifically designed to connect to an Identity store (AD/Samba/LDAP, SQL Database, Web services/application, ...)
 and ease the integration of the Matrix ecosystem with an existing infrastructure, or to build a new one using lasting
 tools.
 
-The core principle of mxisd is to map between Matrix IDs and 3PIDs (Thrid-party Identifiers) for the Homeserver and its
+The core principle of mxisd is to map between Matrix IDs and 3PIDs (Third-party Identifiers) for the Homeserver and its
 users. 3PIDs can be anything that identify a user, like:
 - Full name
 - Email address
@@ -29,8 +29,9 @@ users. 3PIDs can be anything that identify a user, like:
 - Facebook ID
 - ...
 
-mxisd is an enhanced Identity service, which implements the [Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html)
+mxisd is an enhanced Identity service, which implements the
-but also several other features that greatly enhance user experience within Matrix.
+[Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html) but also several
+[other features](#features) that greatly enhance user experience within Matrix.
 
 mxisd is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built as a
 single coherent product.
@@ -64,115 +65,20 @@ currently **cannot be removed**
 - Users can directly find each other using whatever attribute is relevant within your Identity store
 - Federate your Identity lookups so you can discover others and/or others can discover you, all with extensive ACLs
 
-# Quick Start
+# Getting started
-1. [Preparation](#preparation)
+See the [dedicated document](docs/getting-started.md)
-2. [Install](#install)
-3. [Configure](#configure)
-4. [Integrate](#integrate)
-5. [Validate](#validate)
-
-Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
-talk to the central Matrix.org Identity service.  
-This will be a good ground work for further integration with your existing Identity stores.
-
-## Preparation
-You will need:
-- Homeserver
-- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
-
-As synapse requires an HTTPS connection when talking to an Identity service, a reverse proxy is required as mxisd does
-not support HTTPS listener at this time.
-
-For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.  
-You can also use a dedicated domain for mxisd, but will not have access to some features.
-
-Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-io/mxisd/wiki/Gotchas#nating) if you use the same
-hostname.
-
-The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
-If you would like a high-level view of the infrastructure and how each feature is integrated, see the
-[dedicated document](docs/architecture.md)
-
-## Install
-Install via:
-- [Debian package](docs/install/debian.md)
-- [Docker image](docs/install/docker.md)
-- [Sources](docs/build.md)
-
-See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
-
-## Configure
-Create/edit a minimal configuration (see installer doc for the location):
-```
-matrix.domain: 'MyMatrixDomain.org'
-key.path: '/path/to/signing.key.file'
-storage.provider.sqlite.database: '/path/to/mxisd.db'
-```  
-- `matrix.domain` should be set to your Homeserver domain
-- `key.path` will store the signing keys, which must be kept safe!
-- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
-
-If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
-Complete configuration guide is available [here](docs/configure.md).
-
-## Integrate
-For an overview of a typical mxisd infrastructure, see the [dedicated document](docs/architecture.md)
-### Reverse proxy
-#### Apache2
-In the VirtualHost handling the domain with SSL, add the following line and replace `0.0.0.0` by the right address/host.  
-**This line MUST be present before the one for the homeserver!**
-```
-ProxyPass /_matrix/identity/ http://0.0.0.0:8090/_matrix/identity/
-```
-
-Typical VirtualHost configuration would be:
-```
-<VirtualHost *:443>
-    ServerName example.org
-    
-    ...
-    
-    ProxyPreserveHost on
-    ProxyPass /_matrix/identity/ http://10.1.2.3:8090/_matrix/identity/
-    ProxyPass /_matrix/ http://10.1.2.3:8008/_matrix/
-</VirtualHost>
-```
-
-### Synapse
-Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
-In a typical configuration, you would end up with something similair to:
-```
-trusted_third_party_id_servers:
-    - matrix.org
-    - vector.im
-    - example.org
-```
-It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is allowed by synapse. 
-
-### Federation and network discovery
-See the [dedicated document](docs/features/federation.md).
-
-## Validate
-Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
-the relevant hostname which you configured in your reverse proxy.  
-Invite `mxisd-lookup-test@kamax.io` to a room, which should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.  
-**NOTE:** you might not see a Matrix suggestion for the e-mail address, which is normal. Still proceed with the invite.
-  
-If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
-If it did not work, [get in touch](#support) and we'll do our best to get you started.
-
-You can now integrate mxisd further with your infrastructure using the various [features](docs/README.md) guides.
 
 # Support
 ## Community
 If you need help, want to report a bug or just say hi, you can reach us on Matrix at 
-[#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) or [directly peek anonymously](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/).
+[#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) or
+[directly peek anonymously](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/).
 For more high-level discussion about the Identity Server architecture/API, go to 
 [#matrix-identity:matrix.org](https://matrix.to/#/#matrix-identity:matrix.org)
 
 ## Professional
-If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open source technologies/products, 
+If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open
-please visit [our website](https://www.kamax.io/) to get in touch with us and get a quote.
+source technologies/products, please visit [our website](https://www.kamax.io/) to get in touch with us and get a quote.
 
 We offer affordable monthly/yearly support plans for mxisd, synapse or your full Matrix infrastructure.
 
@@ -185,8 +91,8 @@ You can contribute as a community member by:
 - Helping us improve the documentation: tell us what is good or not good (in an issue or in Matrix), or make a PR with
 changes you feel improve the doc.
 - Contribute code directly: we love contributors! All your contributions will be licensed under AGPLv3.
-- Donate! any donation is welcome, regardless how small or big. This will directly be used for the fixed costs and
+- [Donate!](https://liberapay.com/maximusdor/) Any donation is welcome, regardless how small or big, and will directly
-developer time.
+be used for the fixed costs and developer time of mxisd.
 
 You can contribute as an organisation/corporation by:
 - Get a [support contract](#support-professional). This is the best way you can help us as it ensures mxisd is
@@ -194,64 +100,7 @@ maintained regularly and you get direct access to the support team.
 - Sponsoring new features or bug fixes. [Get in touch](#contact) so we can discuss it further.
 
 # FAQ
-### Do I need to use mxisd if I run a Homeserver?
+See the [dedicated document](docs/faq.md)
-No, but it is recommended, even if you don't use any backends or integration.
-
-mxisd in its default configuration will use federation and involve the central Matrix.org Identity servers when
-performing queries, giving you access to at least the same information as if you were not running it.
-
-It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
-privacy consequences, which is not the case with the central Matrix.org servers.
-
-So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
-simple features on top of it.
-
-### I already use the synapse LDAP3 auth provider, why should I care about mxisd?
-The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) only handles on specific flow:
-validate credentials at login.
-
-It does not:
-- Auto-provision user profiles
-- Integrate with Identity management
-- Integrate with Directory searches
-- Protect you against the username case sensitivites issues in synapse
-
-mxisd is a replacement and enhancement of it, and also offers coherent results in all areas, which LDAP3 auth provider
-does not.
-
-### I saw that sydent is the official Identity server implemenation of the Matrix team, I should use that!
-You can, but [sydent](https://github.com/matrix-org/sydent):
-- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
-- is not meant to be linked to a specific Homeserver / domain
-- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
-- forces you to duplicate all your identity data, so people can be found by 3PIDs
-- forces users to enter all their emails and phone numbers manually in their profile
-
-So really, you should go with mxisd.
-
-### I'm not sure I understand what an "Identity server" is supposed to be or do
-The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
-what they want to do with that part of the ecosystem. Therefore, "Identity" is a misleading word currently.  
-Given the scope of the current Identity Service API, it would be best called "Invitation service".
-
-Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
-for groups/corporations/organisation. This is where mxisd comes in.
-
-mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
-up to its "Identity server" name.
-
-### So mxisd is just a big hack! I don't want to use non-official features!
-mxisd primary concern is to always be compatible with the Matrix ecosystem and the Identity service API.  
-Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
-
-We also directly talk with the Matrix developers to ensure all features we implement have their approval, and that we
-are in line with their vision of Identity management within the Matrix ecosystem.
-
-Therefore, using mxisd is a safe choice. It will be like using the central Matrix.org Identity servers, yet not closing
-the door to very nice enhancements and integrations.
-
-### Should I use mxisd if I don't host my own Homeserver?
-No
 
 # Contact
 Get in touch via:

application.example.yaml

@@ -6,6 +6,7 @@
 # Matrix config items #
 #######################
 # Matrix domain, same as the domain configure in your Homeserver configuration.
+# (note: in Synapse Homeserver, the Matrix domain may be defined as 'server_name' in configuration file).
 #
 # This is used to build the various identifiers for identity, auth and directory.
 matrix.domain: ''
@@ -17,10 +18,12 @@ matrix.domain: ''
 # Absolute path for the Identity Server signing key.
 # During testing, /var/tmp/mxisd.key is a possible value
 #
-# For production, use a stable location like:
+# For production, recommended location shall be one of the following:
 #   - /var/opt/mxisd/sign.key
 #   - /var/local/mxisd/sign.key
 #   - /var/lib/mxisd/sign.key
+#
+# The signing key is auto-generated during execution time if not present.
 key.path: ''
 
 
@@ -92,5 +95,5 @@ threepid.medium.email.connectors.smtp.login: "matrix-identity@example.org"
 # Password for the account
 threepid.medium.email.connectors.smtp.password: "ThePassword"
 
-# The e-mail to send as. If empty, will be the same as login
+# The e-mail to send as.
 threepid.medium.email.identity.from: "matrix-identity@example.org"

build.gradle

@@ -47,7 +47,7 @@ String gitVersion() {
     def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")
     ByteArrayOutputStream out = new ByteArrayOutputStream()
     exec {
-        commandLine = ['git', 'describe', '--always', '--dirty']
+        commandLine = ['git', 'describe', '--tags', '--always', '--dirty']
         standardOutput = out
     }
     def v = out.toString().replace(System.lineSeparator(), '')
@@ -74,10 +74,10 @@ dependencies {
     compile 'commons-io:commons-io:2.5'
 
     // Spring Boot - standalone app
-    compile 'org.springframework.boot:spring-boot-starter-web:1.5.3.RELEASE'
+    compile 'org.springframework.boot:spring-boot-starter-web:1.5.10.RELEASE'
 
     // Thymeleaf for HTML templates
-    compile "org.springframework.boot:spring-boot-starter-thymeleaf:1.5.3.RELEASE"
+    compile "org.springframework.boot:spring-boot-starter-thymeleaf:1.5.10.RELEASE"
 
     // Matrix Java SDK
     compile 'io.kamax:matrix-java-sdk:0.0.2'

docs/_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-cayman

docs/architecture.md

@@ -1,6 +1,6 @@
 # Architecture
 ## Overview
-### Basic setup without integration or federation
+### Basic setup without integration or incoming federation
 ```
  Client
    |

docs/backends/README.md

@@ -0,0 +1,5 @@
+# Identity Stores (Backends)
+- [Samba / Active Directory / LDAP](ldap.md)
+- [SQL Databases](sql.md)
+- [Website / Web service / Web app](rest.md)
+- [Google Firebase](firebase.md)

docs/backends/firebase.md

@@ -1,4 +1,14 @@
 # Google Firebase
+https://firebase.google.com/
+
+## Requirements
+This backend requires a suitable Matrix client capable of performing Firebase authentication and passing the following
+information:
+- Firebase User ID as Matrix username
+- Firebase token as Matrix password
+
+If your client is Riot, you will need a custom version.
+
 ## Configuration
 To be completed. For now, see default structure and values:
 ```
@@ -6,4 +16,4 @@ firebase:
   enabled: false
   credentials: '/path/to/firebase/credentials.json'
   database: 'https://my-project.firebaseio.com/'
-```
+```

docs/backends/ldap.md

@@ -1,53 +1,99 @@
-# AD/Samba/LDAP backend
+# LDAP (Samba / Active Directory / OpenLDAP)
+## Getting started
+To use your LDAP backend, add the bare minimum configuration in mxisd config file:
+```
+ldap.enabled: true
+ldap.connection.host: 'ldapHostnameOrIp'
+ldap.connection.bindDn: 'CN=My Mxisd User,OU=Users,DC=example,DC=org'
+ldap.connection.bindPassword: 'TheUserPassword'
+ldap.connection.baseDn: 'OU=Users,DC=example,DC=org'
+```
+These are standard LDAP connection configuration. mxisd will try to connect on port default port 389 without encryption.
+
+---
+
+If you would like to use a TLS/SSL connection, use the following configuration options (STARTLS not supported):
+```
+ldap.connection.tls: true
+ldap.connection.port: 12345
+```
+
+---
+
+You can also set a default global filter on any LDAP queries:
+```
+ldap.filter: '(memberOf=CN=My Matrix Users,OU=Groups,DC=example,DC=org)'
+```
+This example would only return users part of the group called `My Matrix Users`.
+This can be overwritten or append in each specific flow describe below.
+
+---
+
+LDAP features are based on mapping LDAP attributes to Matrix concepts, like a Matrix ID, its localpart, the user display
+name, their email(s) and/or phone number(s).
+     
+Default attributes are well suited for Active Directory/Samba. In case you are using a native LDAP backend, you will
+most certainly configure those mappings.
+
+The following example would set the `uid` attribute as localpart and the Matrix display name to `cn`
+```
+ldap.attribute.uid.type: 'uid'
+ldap.attribute.uid.value: 'uid'
+ldap.attribute.name: 'cn'
+```
+
+You can also change the attribute lists for 3PID, like email or phone numbers.  
+The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67)
+for emails and phone number:
+```
+ldap.attribute.threepid.email:
+  - 'mail'
+  - 'otherMailAttribute'
+
+ldap.attribute.threepid.msisdn:
+  - 'phone'
+  - 'otherPhoneAttribute'
+```
+
+## Identity
+Identity features (related to 3PID invites or searches) are enabled and configured using default values and no specific
+configuration item is needed to get started.
+
+If you would like to overwrite some global configuration relative to filter and/or attributes, see the Identity section
+of the Configuration below.
+
+## Authentication
+No further configuration is needed to enable authentication with LDAP once globally enabled and configured.  
+You have the possiblity to use a different query filter if you wish, see Configuration below.
+
+Profile auto-fill is enabled by default. It will use the `name` and `threepid` configuration options to get a lit of
+attributes to be used to build the user profile to pass on to synapse during authentication.
+
+## Directory
+No further configuration is needed to enable directory with LDAP once globally enabled and configured.
+
+If you would like to use extra attributes in search that are not 3PIDs, like nicknames, group names, employee number:
+```
+ldap.directory.attribute.other:
+  - 'myNicknameAttribute'
+  - 'memberOf'
+  - 'employeeNumberAttribute'
+```
+
 ## Configuration
-### Structure and default values
+Please read the [Configuration](../configure.md) explanatory note if you are not familiar with the terms used below.
-```
+ 
-ldap:
-  enabled: false
-  filter: ''
-  connection:
-    host: ''
-    tls: false
-    port: 389
-    bindDn: ''
-    bindPassword: ''
-    baseDn: ''
-  attribute:
-    uid:
-      type: 'uid'
-      value: 'userPrincipalName'
-    name: 'displayName'
-    threepid:
-      email:
-        - 'mailPrimaryAddress'
-        - 'mail'
-        - 'otherMailbox'
-      msisdn:
-        - 'telephoneNumber'
-        - 'mobile'
-        - 'homePhone'
-        - 'otherTelephone'
-        - 'otherMobile'
-        - 'otherHomePhone'
-  auth:
-    filter: ''
-  directory:
-    attribute:
-      other: []
-    filter: ''
-  identity:
-    filter: ''
-    medium:
-      email: ''
-      msisdn: ''
-```
 ### General
+Base path: `ldap`
+
 | Item      | Description                                                                               |
 |-----------|-------------------------------------------------------------------------------------------|
 | `enabled` | Globaly enable/disable the LDAP backend                                                   |
 | `filter`  | Global filter to apply on all LDAP queries. Can be overwritten in each applicable section |
 
 ### Connection
+Base path: `ldap.connection`
+
 | Item           | Description                                          |
 |----------------|------------------------------------------------------|
 | `host`         | Host to connect to                                   |
@@ -58,6 +104,8 @@ ldap:
 | `baseDn`       | Base DN for queries                                  |
 
 ### Attributes
+Base path: `ldap.attribute`
+
 | Item        | Description                                                                                                            |
 |-------------|------------------------------------------------------------------------------------------------------------------------|
 | `uid.type`  | Indicate how to process the User ID (UID) attribute:                                                                   |
@@ -68,11 +116,15 @@ ldap:
 | `threepid`  | Namespace where each key is a 3PID type and contains a list of attributes                                              |
 
 ### Authentication
+Base path: `ldap.auth`
+
 | Item     | Description                                                                                      |
 |----------|--------------------------------------------------------------------------------------------------|
 | `filter` | Specific user filter applied during authentication. Global filter is used if empty/blank/not set |
 
 ### Directory
+Base path: `ldap.directory`
+
 | Item              | Description                                                         |
 |-------------------|---------------------------------------------------------------------|
 | `attribute.other` | Additional attributes to be used when performing directory searches |
@@ -80,6 +132,8 @@ ldap:
 |                   | Global filter is used if empty/blank/not set                        |
 
 ### Identity
+Base path: `ldap.identity`
+
 | Item     | Description                                                                                       |
 |----------|---------------------------------------------------------------------------------------------------|
 | `filter` | Specific user filter applied during identity search. Global filter is used if empty/blank/not set | 

docs/configure.md

@@ -37,7 +37,7 @@ server:
 **WARNING:** mxisd might overwrite/adapt some values during launch. Those changes will not be reflected into copied keys.
 
 ## Categories
-For each category below, the base configuration path will be given, which needs to be appened to every configuration
+For each category below, the base configuration path will be given, which needs to be appended to every configuration
 item described.
 
 Example: if the base path was `basePath` and the following table was given:

docs/faq.md

@@ -0,0 +1,65 @@
+# FAQ
+### Do I need to use mxisd if I run a Homeserver?
+No, but it is recommended, even if you don't use any backends or integration.
+
+In its default configuration, mxisd will talk to the central Matrix Identity servers and use other federated public
+servers when performing queries, giving you access to at least the same information as if you were not running it.
+
+It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
+privacy consequences, which is not the case with the central Matrix.org servers.
+
+So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
+simple features on top of it.
+
+### I already use the synapse LDAP3 auth provider, why should I care about mxisd?
+The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) is not longer maintained and
+only handles on specific flow: validate credentials at login.
+
+It does not:
+- Auto-provision user profiles
+- Integrate with Identity management
+- Integrate with Directory searches
+- Protect you against the username case sensitivites issues in synapse
+
+mxisd is a replacement and enhancement of it, offering coherent results in all areas, which LDAP3 auth provider
+does not.
+
+### Sydent is the official Identity server implementation of the Matrix team, why not use that?
+You can, but [sydent](https://github.com/matrix-org/sydent):
+- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
+- is not meant to be linked to a specific Homeserver / domain
+- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
+- forces you to duplicate all your identity data, so people can be found by 3PIDs
+- forces users to enter all their emails and phone numbers manually in their profile
+
+So really, you should go with mxisd.
+
+### Will I loose access to the central Matrix.org/Vector.im Identity data if I use mxisd?
+In its default configuration, mxisd act as a proxy to Matrix.org/Vector.im. You will have access to the same data and
+behaviour than if you were using them directly. There is no downside in using mxisd with the default configuration.
+
+mxisd can also be configured not to talk to the central Identity servers if you wish.
+
+### I'm not sure I understand what an "Identity server" is supposed to be or do
+The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
+what they want to do with that part of the ecosystem. Therefore, "Identity" is a misleading word currently.  
+Given the scope of the current Identity Service API, it would be best called "Invitation service".
+
+Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
+for groups/corporations/organisation. This is where mxisd comes in.
+
+mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
+up to its "Identity server" name.
+
+### So mxisd is just a big hack! I don't want to use non-official features!
+mxisd primary concern is to always be compatible with the Matrix ecosystem and the Identity service API.  
+Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
+
+We also directly talk with the Matrix developers to ensure all features we implement have their approval, and that we
+are in line with their vision of Identity management within the Matrix ecosystem.
+
+Therefore, using mxisd is a safe choice. It will be like using the central Matrix.org Identity servers, yet not closing
+the door to very nice enhancements and integrations.
+
+### Should I use mxisd if I don't host my own Homeserver?
+No

docs/features/authentication.md

@@ -1,8 +1,31 @@
 # Authentication
-Performed via [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)  
+
-Point the `endpoint` to mxisd internal IP on port 8090
+- [Description](#description)
+- [Overview](#overview)
+- [Getting started](#getting-started)
+  - [Synapse](#synapse)
+  - [mxisd](#mxisd)
+  - [Validate](#validate)
+- [Next steps](#next-steps)
+  - [Profile auto-fil](#profile-auto-fill)
+- [Advanced Authentication](#advanced-authentication)
+  - [Requirements](#requirements)
+  - [Configuration](#configuration)
+    - [Reverse Proxy](#reverse-proxy)
+      - [Apache2](#apache2)
+    - [DNS Overwrite](#dns-overwrite)
+    - [Backends](#backends) 
+
+## Description
+Authentication is an enhanced Identity feature of mxisd to ensure coherent and centralized identity management.
+
+It allows to use Identity stores configured in mxisd to authenticate users on your Homeserver.
+
+This feature can also provide the ability to users to login on the Homeserver using their third party identities (3PIDs) provided by an Identity store.
 
 ## Overview
+An overview of the Authentication process is depicted below: 
+
 ```
                                                                                     Backends
  Client                                                                             +------+
@@ -10,7 +33,7 @@ Point the `endpoint` to mxisd internal IP on port 8090
    |   +---------------+  /_matrix/identity     | mxisd                   |    |    +------+
    +-> | Reverse proxy | >------------------+   |                         |    |
        +--|------------+                    |   |                         |    |    +--------+
-          |                                 +-----> Check wiht backends >------+--> | SQL DB |
+          |                                 +-----> Check with backends >------+--> | SQL DB |
      Login request                          |   |                         |    |    +--------+
           |                                 |   |     |                   |    |
           |   +--------------------------+  |   +-----|-------------------+    +-->  Others
@@ -23,6 +46,126 @@ Point the `endpoint` to mxisd internal IP on port 8090
               |   user profiles          |    If valid credentials and supported by backend
               +--------------------------+
 ```
+Performed on [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)
+
+## Getting started
+Authentication is possible by linking synapse and mxisd together using the REST auth module
+(also known as password provider).
+
+### Synapse
+- Install the [REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth).
+- Edit your synapse configuration:
+  - As described by the auth module documentation
+  - Set `endpoint` to `http://mxisdAddress:8090` - Replace `mxisdAddress` by an IP/host name that provides a direct
+  connection to mxisd.  
+  This **MUST NOT** be a public address, and SHOULD NOT go through a reverse proxy.
+- Restart synapse
+
+### mxisd
+- Configure and enable at least one [Identity store](../backends/)
+- Restart mxisd
+
+### Validate
+Login on the Homeserver using credentials present in your backend.
+
+## Next steps
+### Profile auto-fill
+Auto-filling user profile depends on two conditions:
+- The REST auth module is configured for it, which is the case by default
+- Your Identity store is configured to provide profile data. See your Identity store [documentation](../backends/) on
+how to enable the feature.
+
+
+## Advanced Authentication
+The Authentication feature allows users to login to their Homeserver by using their 3PIDs registered in an available Identity store.
+
+This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/login` as depicted below:
+
+```
+            +----------------------------+
+            |  Reverse Proxy             |
+            |                            |
+            |                            |     Step 1    +---------------------------+     Step 2
+            |                            |               |                           |
+Client+---->| /_matrix/client/r0/login +---------------->|                           | Look up address  +---------+
+            |                      ^     |               |  mxisd - Identity server  +----------------->| Backend |
+            |                      |     |               |                           |                  +---------+
+            | /_matrix/* +--+      +---------------------+                           |
+            |               |            |               +---------------+-----------+
+            |               |            |     Step 4                    |
+            |               |            |                               | Step 3
+            +---------------|------------+                               |
+                            |                                            | /_matrix/client/r0/login
+                            |                       +--------------+     |
+                            |                       |              |     |
+                            +---------------------->|  Homeserver  |<----+
+                                                    |              |
+                                                    +--------------+
+
+```
+
+Steps of user authentication using a 3PID:
+1. The intercepted login request is directly sent to mxisd instead of the Homeserver.
+2. Enabled backends are queried for a matching user identity in order to modify the request to use the user name.
+3. The Homeserver, from which the request was intercepted, is queried using the request at previous step. Its address is resolved using the DNS Overwrite feature to reach its internal address on a non-encrypted port.
+4. The response from the Homeserver is sent back to the client, believing it was the HS which directly answered.
+
+### Requirements
+- Reverse proxy setup
+- Homeserver
+- Compatible Identity backends:
+	- LDAP
+	- SQL
+	- REST
+
+### Configuration
+
+#### Reverse Proxy
+
+##### Apache2
+The specific configuration to put under the relevant `VirtualHost`:
+```
+ProxyPass /_matrix/client/r0/login http://localhost:8090/_matrix/client/r0/login
+```
+`ProxyPreserveHost` or equivalent must be enabled to detect to which Homeserver mxisd should talk to when building results.
+
+Your VirtualHost should now look like this:
+```
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/client/r0/login http://localhost:8090/_matrix/client/r0/login
+    ProxyPass /_matrix/identity/ http://localhost:8090/_matrix/identity/
+    ProxyPass /_matrix/ http://localhost:8008/_matrix/
+</VirtualHost>
+```
+
+#### DNS Overwrite
+Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
+
+
+To do so, put the following configuration in your `application.yaml`:
+```
+dns.overwrite.homeserver.client:
+  - name: 'example.org'
+    value: 'http://localhost:8008'
+```
+`name` must be the hostname of the URL that clients use when connecting to the Homeserver.
+In case the hostname is the same as your Matrix domain, you can use `${matrix.domain}` to auto-populate the `value` using the `matrix.domain` configuration option and avoid duplicating it.
+
+value is the base internal URL of the Homeserver, without any /_matrix/.. or trailing /.
+
+#### Backends
+The Backends should be configured as described in the documentation of the [Directory User](directory-users.md) feature. 
+
+
+
+
+
+
+
+
 
-## Profile auto-fill
-To be documented

docs/features/directory-users.md

@@ -4,11 +4,14 @@
 - [Requirements](#requirements)
 - [Configuration](#configuration)
   - [Reverse Proxy](#reverse-proxy)
+    - [Apache2](#apache2)
+    - [nginx](#nginx)
   - [DNS Overwrite](#dns-overwrite)
   - [Backends](#backends)
     - [LDAP](#ldap)
     - [SQL](#sql)
     - [REST](#rest)
+- [Next steps](#next-steps)
 
 ## Description
 This feature allows you to search for existing and/or potential users that are already present in your Identity backend
@@ -61,16 +64,66 @@ which directly answered the request.
   
 ## Configuration
 ### Reverse Proxy
-Apache2 configuration to put under the relevant virtual domain:
+#### Apache2
+The specific configuration to put under the relevant `VirtualHost`:
 ```
-ProxyPreserveHost on
+ProxyPass /_matrix/client/r0/user_directory/ http://0.0.0.0:8090/_matrix/client/r0/user_directory/
-ProxyPass /_matrix/identity/ http://mxisdInternalIpAddress:8090/_matrix/identity/
-ProxyPass /_matrix/client/r0/user_directory/ http://mxisdInternalIpAddress:8090/_matrix/client/r0/user_directory/
-ProxyPass /_matrix/ http://HomeserverInternalIpAddress:8008/_matrix/
 ```
 `ProxyPreserveHost` or equivalent must be enabled to detect to which Homeserver mxisd should talk to when building
 results.
 
+Your `VirtualHost` should now look like this:
+```
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/client/r0/user_directory/ http://localhost:8090/_matrix/client/r0/user_directory/
+    ProxyPass /_matrix/identity/ http://localhost:8090/_matrix/identity/
+    ProxyPass /_matrix/ http://localhost:8008/_matrix/
+</VirtualHost>
+```
+
+#### nginx
+The specific configuration to add under your `server` section is:
+```
+location /_matrix/client/r0/user_directory {
+    proxy_pass http://0.0.0.0:8090/_matrix/client/r0/user_directory;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $remote_addr;
+}
+```
+
+Your `server` section should now look like this:
+```
+server {
+    listen 443 ssl;
+    server_name example.org;
+    
+    ...
+    
+    location /_matrix/client/r0/user_directory {
+        proxy_pass http://localhost:8090/_matrix/client/r0/user_directory;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
 ### DNS Overwrite
 Just like you need to configure a reverse proxy to send client requests to mxisd, you also need to configure mxisd with
 the internal IP of the Homeserver so it can talk to it directly to integrate its directory search.
@@ -165,3 +218,11 @@ For each query, `type` can be used to tell mxisd how to process the ID column:
 
 #### REST
 See the [dedicated document](../backends/rest.md)
+
+## Next steps
+### Homeserver results
+You can configure if the Homeserver should be queried at all when doing a directory search.  
+To disable Homeserver results, set the following in mxisd config file:
+```
+directory.exclude.homeserever: true
+```

docs/features/identity.md

@@ -1,3 +1,18 @@
-To be documented.
+# Matrix Identity Service
+**WARNING**: This document is incomplete and can be missleading.
 
-Implementation of the [Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html)
+Implementation of the [Unofficial Matrix Identity Service API](https://kamax.io/matrix/api/identity_service/unstable.html).
+
+## Invitation
+Resolution can be customized using the following configuration:
+
+`invite.resolution.recursive`  
+- Default value: `true`  
+- Description: Control if the pending invite resolution should be done recursively or not.  
+  **DANGER ZONE:** This setting has the potential to create "an isolated island", which can have unexpected side effects
+  and break invites in rooms. This will most likely not have the effect you think it does. Only change the value if you
+  understand the consequences.
+
+`invite.resolution.timer`  
+- Default value: `1`  
+- Description: How often, in minutes, mxisd should try to resolve pending invites.

docs/getting-started.md

@@ -0,0 +1,145 @@
+# Getting started
+1. [Preparation](#preparation)
+2. [Install](#install)
+3. [Configure](#configure)
+4. [Integrate](#integrate)
+5. [Validate](#validate)
+6. [Next steps](#next-steps)
+
+Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
+talk to the central Matrix.org Identity service.  
+This will be a good ground work for further integration with your existing Identity stores.
+
+## Preparation
+You will need:
+- Homeserver
+- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
+
+As synapse requires an HTTPS connection when talking to an Identity service, a reverse proxy is required as mxisd does
+not support HTTPS listener at this time.
+
+For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.  
+You can also use a dedicated domain for mxisd, but will not have access to some features.
+
+Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-io/mxisd/wiki/Gotchas#nating) if you use the same
+hostname.
+
+The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
+If you would like a high-level view of the infrastructure and how each feature is integrated, see the
+[dedicated document](architecture.md)
+
+## Install
+Install via:
+- [Debian package](install/debian.md)
+- [Docker image](install/docker.md)
+- [Sources](build.md)
+
+See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
+
+## Configure
+**NOTE**: please view the install instruction for your platform, as this step might be optional/handled for you.
+
+Create/edit a minimal configuration (see installer doc for the location):
+```
+matrix.domain: 'MyMatrixDomain.org'
+key.path: '/path/to/signing.key.file'
+storage.provider.sqlite.database: '/path/to/mxisd.db'
+```  
+- `matrix.domain` should be set to your Homeserver domain
+- `key.path` will store the signing keys, which must be kept safe!
+- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
+
+If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
+Complete configuration guide is available [here](configure.md).
+
+## Integrate
+For an overview of a typical mxisd infrastructure, see the [dedicated document](architecture.md)
+### Reverse proxy
+#### Apache2
+In the `VirtualHost` section handling the domain with SSL, add the following and replace `0.0.0.0` by the internal
+hostname/IP pointing to mxisd.  
+**This line MUST be present before the one for the homeserver!**
+```
+ProxyPass /_matrix/identity/ http://0.0.0.0:8090/_matrix/identity/
+```
+
+Typical configuration would look like:
+```
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/identity/ http://localhost:8090/_matrix/identity/
+    ProxyPass /_matrix/ http://localhost:8008/_matrix/
+</VirtualHost>
+```
+
+#### nginx
+In the `server` section handling the domain with SSL, add the following and replace `0.0.0.0` with the internal
+hostname/IP pointing to mxisd.
+**This line MUST be present before the one for the homeserver!**
+```
+location /_matrix/identity {
+    proxy_pass http://0.0.0.0:8090/_matrix/identity;
+}
+```
+
+Typical configuration would look like:
+```
+server {
+    listen 443 ssl;
+    server_name example.org;
+    
+    ...
+    
+    location /_matrix/identity {
+        proxy_pass http://localhost:8090/_matrix/identity;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+    
+    location /_matrix {
+        proxy_pass http://localhost:8008/_matrix;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $remote_addr;
+    }
+}
+```
+
+### Synapse
+Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
+In a typical configuration, you would end up with something similair to:
+```
+trusted_third_party_id_servers:
+    - matrix.org
+    - vector.im
+    - example.org
+```
+It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is authoritative for your HS.
+
+## Validate
+Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
+the relevant hostname which you configured in your reverse proxy.  
+Invite `mxisd-lookup-test@kamax.io` to a room, which should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.  
+**NOTE:** you might not see a Matrix suggestion for the e-mail address, which is normal. Still proceed with the invite.
+  
+If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
+If it did not work, [get in touch](#support) and we'll do our best to get you started.
+
+You can now integrate mxisd further with your infrastructure using the various [features](README.md) guides.
+
+## Next steps
+Once your mxisd server is up and running, here are the next steps to further enhance and integrate your installation:
+
+Enable extra features:
+- [Federation](features/federation.md)
+- [Authenticate with synapse](features/authentication.md), profile auto-provisioning if you wish
+- [Directory search](features/directory-users.md)
+
+Use your Identity stores:
+- [LDAP / Samba / Active directory](backends/ldap.md)
+- [SQL Database](backends/sql.md)
+- [Website / Web service / Web app](backends/rest.md)
+- [Google Firebase](backends/firebase.md)

docs/install/docker.md

@@ -5,10 +5,18 @@ Pull the latest stable image:
 docker pull kamax/mxisd
 ```
 
+## Configure
+On first run, simply using `MATRIX_DOMAIN` as an environment variable will create a default config for you.  
+You can also provide a configuration file named `mxisd.yaml` in the volume mapped to `/etc/mxisd` before starting your
+container.
+
 ## Run
-Run it (adapt volume paths to your host):
+Use the following command after adapting to your needs:
+- The `MATRIX_DOMAIN` environment variable to yours
+- The volumes host paths
+
 ```
-docker run --rm -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
+docker run --rm -e MATRIX_DOMAIN=example.org -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
 ```
 
 For more info, including the list of possible tags, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)

docs/sessions/3pid.md

@@ -276,14 +276,14 @@ session:
 **IMPORTANT**: When using local-only mode, you will also need to link mxisd to synapse if you want user searches and invites to work.
 To do so, add/edit the following configuration keys:
 ```
-sql:
+synapseSql:
   enabled: true
-  type: 'postgresql'
+  type: 'SET TO PROPER VALUE'
-  connection: ''
+  connection: 'SET TO PROPER VALUE'
 ```
-- `sql.enabled` set to `true` to activate the SQL backend.
+- `synapseSql.enabled` set to `true` to activate the SQL backend.
-- `sql.type` can be set to `sqlite` or `postgresql`, depending on your synapse setup.
+- `synapseSql.type` can be set to `sqlite` or `postgresql`, depending on your synapse setup.
-- `sql.connection` use a JDBC format which is appened after the `jdbc:type:` connection URI.
+- `synapseSql.connection` use a JDBC format which is appened after the `jdbc:type:` connection URI.
 Example values for each type:
   - `sqlite`: `/path/to/homeserver.db`
   - `postgresql`: `//localhost/database?user=synapse&password=synapse`

src/docker/start.sh

@@ -1,2 +1,25 @@
-#!/bin/sh
+#!/usr/bin/env bash
-exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.config.location=/etc/mxisd/ -Dspring.config.name=mxisd -jar /mxisd.jar
+if [[ -n "$CONF_FILE_PATH" ]] && [ ! -f "$CONF_FILE_PATH" ]; then
+    echo "Generating config file $CONF_FILE_PATH"
+    touch "CONF_FILE_PATH"
+
+    if [[ -n "$MATRIX_DOMAIN" ]]; then
+        echo "Setting matrix domain to $MATRIX_DOMAIN"
+        echo "matrix.domain: $MATRIX_DOMAIN" >> "$CONF_FILE_PATH"
+    fi
+
+    if [[ -n "$SIGN_KEY_PATH" ]]; then
+        echo "Setting signing key path to $SIGN_KEY_PATH"
+        echo "key.path: $SIGN_KEY_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    if [[ -n "$SQLITE_DATABASE_PATH" ]]; then
+        echo "Setting SQLite DB path to $SQLITE_DATABASE_PATH"
+        echo "storage.provider.sqlite.database: $SQLITE_DATABASE_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    echo "Starting mxisd..."
+    echo
+fi
+
+exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.config.location=/etc/mxisd/ -Dspring.config.name=mxisd -jar /mxisd.jar

src/main/java/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java

@@ -20,12 +20,17 @@
 
 package io.kamax.mxisd.backend.ldap;
 
+import com.google.i18n.phonenumbers.NumberParseException;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import io.kamax.matrix.ThreePidMedium;
 import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.ThreePid;
 import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.ldap.LdapConfig;
+import io.kamax.mxisd.config.ldap.generic.GenericLdapConfig;
+import io.kamax.mxisd.util.GsonUtil;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
@@ -41,14 +46,20 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;
 
 import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
 
 @Component
-public class LdapAuthProvider extends LdapGenericBackend implements AuthenticatorProvider {
+public class LdapAuthProvider extends LdapBackend implements AuthenticatorProvider {
 
     private Logger log = LoggerFactory.getLogger(LdapAuthProvider.class);
 
+    private PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+
     @Autowired
-    public LdapAuthProvider(LdapConfig cfg, MatrixConfig mxCfg) {
+    public LdapAuthProvider(GenericLdapConfig cfg, MatrixConfig mxCfg) {
         super(cfg, mxCfg);
     }
 
@@ -57,6 +68,21 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
         return getCfg().isEnabled();
     }
 
+    private Optional<String> getMsisdn(String phoneNumber) {
+        try { // FIXME export into dedicated ThreePid class within SDK (copy from Firebase Auth)
+            return Optional.of(phoneUtil.format(
+                    phoneUtil.parse(
+                            phoneNumber,
+                            null // No default region
+                    ),
+                    PhoneNumberUtil.PhoneNumberFormat.E164
+            ).substring(1)); // We want without the leading +
+        } catch (NumberParseException e) {
+            log.warn("Invalid phone number: {}", phoneNumber);
+            return Optional.empty();
+        }
+    }
+
     @Override
     public BackendAuthResult authenticate(_MatrixID mxid, String password) {
         log.info("Performing auth for {}", mxid);
@@ -66,7 +92,7 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
             bind(conn);
 
             String uidType = getAt().getUid().getType();
-            String userFilterValue = StringUtils.equals(LdapGenericBackend.UID, uidType) ? mxid.getLocalPart() : mxid.getId();
+            String userFilterValue = StringUtils.equals(LdapBackend.UID, uidType) ? mxid.getLocalPart() : mxid.getId();
             if (StringUtils.isBlank(userFilterValue)) {
                 log.warn("Username is empty, failing auth");
                 return BackendAuthResult.failure();
@@ -74,7 +100,19 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
 
             String userFilter = "(" + getUidAtt() + "=" + userFilterValue + ")";
             userFilter = buildWithFilter(userFilter, getCfg().getAuth().getFilter());
-            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, getUidAtt(), getAt().getName())) {
+
+            Set<String> attributes = new HashSet<>();
+            attributes.add(getUidAtt());
+            attributes.add(getAt().getName());
+            getAt().getThreepid().forEach((k, v) -> attributes.addAll(v));
+            String[] attArray = new String[attributes.size()];
+            attributes.toArray(attArray);
+
+            log.debug("Base DN: {}", getBaseDn());
+            log.debug("Query: {}", userFilter);
+            log.debug("Attributes: {}", GsonUtil.build().toJson(attArray));
+
+            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, attArray)) {
                 while (cursor.next()) {
                     Entry entry = cursor.get();
                     String dn = entry.getDn().getName();
@@ -99,7 +137,24 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
                     log.info("DN {} is a valid match", dn);
 
                     // TODO should we canonicalize the MXID?
-                    return BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                    BackendAuthResult result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                    log.info("Processing 3PIDs for profile");
+                    getAt().getThreepid().forEach((k, v) -> {
+                        log.info("Processing 3PID type {}", k);
+                        v.forEach(attId -> {
+                            List<String> values = getAttributes(entry, attId);
+                            log.info("\tAttribute {} has {} value(s)", attId, values.size());
+                            getAttributes(entry, attId).forEach(tpidValue -> {
+                                if (ThreePidMedium.PhoneNumber.is(k)) {
+                                    tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
+                                }
+                                result.withThreePid(new ThreePid(k, tpidValue));
+                            });
+                        });
+                    });
+
+                    log.info("Found {} 3PIDs", result.getProfile().getThreePids().size());
+                    return result;
                 }
             } catch (CursorLdapReferralException e) {
                 log.warn("Entity for {} is only available via referral, skipping", mxid);

src/main/java/io/kamax/mxisd/backend/ldap/LdapBackend.java

@@ -21,10 +21,10 @@
 package io.kamax.mxisd.backend.ldap;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.ldap.LdapAttributeConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.entry.Attribute;
+import org.apache.directory.api.ldap.model.entry.AttributeUtils;
 import org.apache.directory.api.ldap.model.entry.Entry;
 import org.apache.directory.api.ldap.model.exception.LdapException;
 import org.apache.directory.ldap.client.api.LdapConnection;
@@ -32,21 +32,24 @@ import org.apache.directory.ldap.client.api.LdapNetworkConnection;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Optional;
 
-public abstract class LdapGenericBackend {
+public abstract class LdapBackend {
 
     public static final String UID = "uid";
     public static final String MATRIX_ID = "mxid";
 
-    private Logger log = LoggerFactory.getLogger(LdapGenericBackend.class);
+    private Logger log = LoggerFactory.getLogger(LdapBackend.class);
 
     private LdapConfig cfg;
     private MatrixConfig mxCfg;
 
-    public LdapGenericBackend(LdapConfig cfg, MatrixConfig mxCfg) {
+    public LdapBackend(LdapConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.mxCfg = mxCfg;
     }
@@ -56,10 +59,10 @@ public abstract class LdapGenericBackend {
     }
 
     protected String getBaseDn() {
-        return cfg.getConn().getBaseDn();
+        return cfg.getConnection().getBaseDn();
     }
 
-    protected LdapAttributeConfig getAt() {
+    protected LdapConfig.Attribute getAt() {
         return cfg.getAttribute();
     }
 
@@ -68,14 +71,14 @@ public abstract class LdapGenericBackend {
     }
 
     protected synchronized LdapConnection getConn() throws LdapException {
-        return new LdapNetworkConnection(cfg.getConn().getHost(), cfg.getConn().getPort(), cfg.getConn().isTls());
+        return new LdapNetworkConnection(cfg.getConnection().getHost(), cfg.getConnection().getPort(), cfg.getConnection().isTls());
     }
 
     protected void bind(LdapConnection conn) throws LdapException {
-        if (StringUtils.isBlank(cfg.getConn().getBindDn()) && StringUtils.isBlank(cfg.getConn().getBindPassword())) {
+        if (StringUtils.isBlank(cfg.getConnection().getBindDn()) && StringUtils.isBlank(cfg.getConnection().getBindPassword())) {
             conn.anonymousBind();
         } else {
-            conn.bind(cfg.getConn().getBindDn(), cfg.getConn().getBindPassword());
+            conn.bind(cfg.getConnection().getBindDn(), cfg.getConnection().getBindPassword());
         }
     }
 
@@ -124,7 +127,6 @@ public abstract class LdapGenericBackend {
     public Optional<String> getAttribute(Entry entry, String attName) {
         Attribute attribute = entry.get(attName);
         if (attribute == null) {
-            log.info("DN {}: no attribute {}, skipping", entry.getDn(), attName);
             return Optional.empty();
         }
 
@@ -137,4 +139,22 @@ public abstract class LdapGenericBackend {
         return Optional.of(value);
     }
 
+    public List<String> getAttributes(Entry entry, String attName) {
+        List<String> values = new ArrayList<>();
+        javax.naming.directory.Attribute att = AttributeUtils.toAttributes(entry).get(attName);
+        if (att == null) {
+            return values;
+        }
+
+        try {
+            NamingEnumeration<?> list = att.getAll();
+            while (list.hasMore()) {
+                values.add(list.next().toString());
+            }
+        } catch (NamingException e) {
+            log.warn("Error while processing LDAP attribute {}, result could be incomplete!", attName, e);
+        }
+        return values;
+    }
+
 }

src/main/java/io/kamax/mxisd/backend/ldap/LdapDirectoryProvider.java

@@ -21,11 +21,12 @@
 package io.kamax.mxisd.backend.ldap;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.ldap.LdapAttributeConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
+import io.kamax.mxisd.config.ldap.generic.GenericLdapConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.IDirectoryProvider;
 import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.util.GsonUtil;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
 import org.apache.directory.api.ldap.model.cursor.EntryCursor;
@@ -43,12 +44,12 @@ import java.util.ArrayList;
 import java.util.List;
 
 @Component
-public class LdapDirectoryProvider extends LdapGenericBackend implements IDirectoryProvider {
+public class LdapDirectoryProvider extends LdapBackend implements IDirectoryProvider {
 
     private Logger log = LoggerFactory.getLogger(LdapDirectoryProvider.class);
 
     @Autowired
-    public LdapDirectoryProvider(LdapConfig cfg, MatrixConfig mxCfg) {
+    public LdapDirectoryProvider(GenericLdapConfig cfg, MatrixConfig mxCfg) {
         super(cfg, mxCfg);
     }
 
@@ -64,15 +65,18 @@ public class LdapDirectoryProvider extends LdapGenericBackend implements IDirect
         try (LdapConnection conn = getConn()) {
             bind(conn);
 
-            LdapAttributeConfig atCfg = getCfg().getAttribute();
+            LdapConfig.Attribute atCfg = getCfg().getAttribute();
 
             attributes = new ArrayList<>(attributes);
             attributes.add(getUidAtt());
             String[] attArray = new String[attributes.size()];
             attributes.toArray(attArray);
-
             String searchQuery = buildOrQueryWithFilter(getCfg().getDirectory().getFilter(), "*" + query + "*", attArray);
+
+            log.debug("Base DN: {}", getBaseDn());
             log.debug("Query: {}", searchQuery);
+            log.debug("Attributes: {}", GsonUtil.build().toJson(attArray));
+
             try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, attArray)) {
                 while (cursor.next()) {
                     Entry entry = cursor.get();

src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java

@@ -21,12 +21,13 @@
 package io.kamax.mxisd.backend.ldap;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.ldap.LdapConfig;
+import io.kamax.mxisd.config.ldap.generic.GenericLdapConfig;
 import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import io.kamax.mxisd.util.GsonUtil;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
 import org.apache.directory.api.ldap.model.cursor.CursorLdapReferralException;
 import org.apache.directory.api.ldap.model.cursor.EntryCursor;
@@ -44,11 +45,11 @@ import java.util.List;
 import java.util.Optional;
 
 @Component
-public class LdapThreePidProvider extends LdapGenericBackend implements IThreePidProvider {
+public class LdapThreePidProvider extends LdapBackend implements IThreePidProvider {
 
     private Logger log = LoggerFactory.getLogger(LdapThreePidProvider.class);
 
-    public LdapThreePidProvider(LdapConfig cfg, MatrixConfig mxCfg) {
+    public LdapThreePidProvider(GenericLdapConfig cfg, MatrixConfig mxCfg) {
         super(cfg, mxCfg);
     }
 
@@ -68,13 +69,20 @@ public class LdapThreePidProvider extends LdapGenericBackend implements IThreePi
     }
 
     private Optional<String> lookup(LdapConnection conn, String medium, String value) {
-        Optional<String> queryOpt = getCfg().getIdentity().getQuery(medium);
+        Optional<String> tPidQueryOpt = getCfg().getIdentity().getQuery(medium);
-        if (!queryOpt.isPresent()) {
+        if (!tPidQueryOpt.isPresent()) {
             log.warn("{} is not a configured 3PID type for LDAP lookup", medium);
             return Optional.empty();
         }
 
-        String searchQuery = queryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        // we merge 3PID specific query with global/specific filter, if one exists.
+        String tPidQuery = tPidQueryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        String searchQuery = buildWithFilter(tPidQuery, getCfg().getIdentity().getFilter());
+
+        log.debug("Base DN: {}", getBaseDn());
+        log.debug("Query: {}", searchQuery);
+        log.debug("Attributes: {}", GsonUtil.build().toJson(getUidAtt()));
+
         try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, getUidAtt())) {
             while (cursor.next()) {
                 Entry entry = cursor.get();

src/main/java/io/kamax/mxisd/backend/ldap/netiq/NetIqLdapAuthProvider.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.ldap.netiq;
+
+import io.kamax.mxisd.backend.ldap.LdapAuthProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.netiq.NetIqLdapConfig;
+import org.springframework.stereotype.Component;
+
+@Component
+public class NetIqLdapAuthProvider extends LdapAuthProvider {
+
+    public NetIqLdapAuthProvider(NetIqLdapConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    // FIXME this is duplicated in the other NetIQ classes, due to the Matrix ID generation code that was not abstracted
+    @Override
+    public String buildMatrixIdFromUid(String uid) {
+        return super.buildMatrixIdFromUid(uid).toLowerCase();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/ldap/netiq/NetIqLdapDirectoryProvider.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.ldap.netiq;
+
+import io.kamax.mxisd.backend.ldap.LdapDirectoryProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.netiq.NetIqLdapConfig;
+import org.springframework.stereotype.Component;
+
+@Component
+public class NetIqLdapDirectoryProvider extends LdapDirectoryProvider {
+
+    public NetIqLdapDirectoryProvider(NetIqLdapConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    // FIXME this is duplicated in the other NetIQ classes, due to the Matrix ID generation code that was not abstracted
+    @Override
+    public String buildMatrixIdFromUid(String uid) {
+        return super.buildMatrixIdFromUid(uid).toLowerCase();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/ldap/netiq/NetIqLdapThreePidProvider.java

@@ -0,0 +1,41 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.ldap.netiq;
+
+import io.kamax.mxisd.backend.ldap.LdapThreePidProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.netiq.NetIqLdapConfig;
+import org.springframework.stereotype.Component;
+
+@Component
+public class NetIqLdapThreePidProvider extends LdapThreePidProvider {
+
+    public NetIqLdapThreePidProvider(NetIqLdapConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+    // FIXME this is duplicated in the other NetIQ classes, due to the Matrix ID generation code that was not abstracted
+    @Override
+    public String buildMatrixIdFromUid(String uid) {
+        return super.buildMatrixIdFromUid(uid).toLowerCase();
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/memory/MemoryIdentityStore.java

@@ -0,0 +1,111 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.memory;
+
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix.ThreePid;
+import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.UserIdType;
+import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.memory.MemoryIdentityConfig;
+import io.kamax.mxisd.config.memory.MemoryStoreConfig;
+import io.kamax.mxisd.config.memory.MemoryThreePid;
+import io.kamax.mxisd.lookup.SingleLookupReply;
+import io.kamax.mxisd.lookup.SingleLookupRequest;
+import io.kamax.mxisd.lookup.ThreePidMapping;
+import io.kamax.mxisd.lookup.provider.IThreePidProvider;
+import org.apache.commons.lang.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+import java.util.Collections;
+import java.util.List;
+import java.util.Optional;
+
+@Component
+public class MemoryIdentityStore implements AuthenticatorProvider, IThreePidProvider {
+
+    private final Logger logger = LoggerFactory.getLogger(MemoryIdentityStore.class);
+
+    private final MatrixConfig mxCfg;
+    private final MemoryStoreConfig cfg;
+
+    @Autowired
+    public MemoryIdentityStore(MatrixConfig mxCfg, MemoryStoreConfig cfg) {
+        this.mxCfg = mxCfg;
+        this.cfg = cfg;
+    }
+
+    public Optional<MemoryIdentityConfig> findByUsername(String username) {
+        return cfg.getIdentities().stream().filter(id -> StringUtils.equals(id.getUsername(), username)).findFirst();
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return cfg.isEnabled();
+    }
+
+    @Override
+    public boolean isLocal() {
+        return true;
+    }
+
+    @Override
+    public int getPriority() {
+        return Integer.MAX_VALUE;
+    }
+
+    @Override
+    public Optional<SingleLookupReply> find(SingleLookupRequest request) {
+        logger.info("Performing lookup {} of type {}", request.getThreePid(), request.getType());
+        ThreePid req = new ThreePid(request.getType(), request.getThreePid());
+        for (MemoryIdentityConfig id : cfg.getIdentities()) {
+            for (MemoryThreePid threepid : id.getThreepids()) {
+                if (req.equals(new ThreePid(threepid.getMedium(), threepid.getAddress()))) {
+                    return Optional.of(new SingleLookupReply(request, new MatrixID(id.getUsername(), mxCfg.getDomain())));
+                }
+            }
+        }
+
+        return Optional.empty();
+    }
+
+    @Override
+    public List<ThreePidMapping> populate(List<ThreePidMapping> mappings) {
+        return Collections.emptyList();
+    }
+
+    @Override
+    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
+        return findByUsername(mxid.getLocalPart()).map(id -> {
+            if (!StringUtils.equals(id.getUsername(), mxid.getLocalPart())) {
+                return BackendAuthResult.failure();
+            } else {
+                return BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, "");
+            }
+        }).orElseGet(BackendAuthResult::failure);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/rest/RestDirectoryProvider.java

@@ -31,10 +31,12 @@ import io.kamax.mxisd.util.RestClientUtils;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang.StringUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
+import org.springframework.stereotype.Component;
 
 import java.io.IOException;
 import java.nio.charset.StandardCharsets;
 
+@Component
 public class RestDirectoryProvider extends RestProvider implements IDirectoryProvider {
 
     private MatrixConfig mxCfg;

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlAuthProvider.java

@@ -24,7 +24,7 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.ServerConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.invitation.InvitationManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -32,15 +32,15 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;
 
 @Component
-public class SqlAuthProvider implements AuthenticatorProvider {
+public class GenericSqlAuthProvider implements AuthenticatorProvider {
 
-    private Logger log = LoggerFactory.getLogger(SqlAuthProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlAuthProvider.class);
 
     @Autowired
     private ServerConfig srvCfg;
 
     @Autowired
-    private SqlProviderConfig cfg;
+    private GenericSqlProviderConfig cfg;
 
     @Autowired
     private InvitationManager invMgr;

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlDirectoryProvider.java

@@ -22,8 +22,8 @@ package io.kamax.mxisd.backend.sql;
 
 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.SqlConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.IDirectoryProvider;
 import io.kamax.mxisd.exception.InternalServerError;
@@ -39,16 +39,16 @@ import java.util.Optional;
 
 import static io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult.Result;
 
-public abstract class SqlDirectoryProvider implements IDirectoryProvider {
+public abstract class GenericSqlDirectoryProvider implements IDirectoryProvider {
 
-    private Logger log = LoggerFactory.getLogger(SqlDirectoryProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlDirectoryProvider.class);
 
     protected SqlConfig cfg;
     private MatrixConfig mxCfg;
 
     private SqlConnectionPool pool;
 
-    public SqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
+    public GenericSqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.pool = new SqlConnectionPool(cfg);
         this.mxCfg = mxCfg;
@@ -72,7 +72,7 @@ public abstract class SqlDirectoryProvider implements IDirectoryProvider {
         return Optional.of(item);
     }
 
-    public UserDirectorySearchResult search(String searchTerm, SqlProviderConfig.Query query) {
+    public UserDirectorySearchResult search(String searchTerm, GenericSqlProviderConfig.Query query) {
         try (Connection conn = pool.get()) {
             log.info("Will execute query: {}", query.getValue());
             try (PreparedStatement stmt = conn.prepareStatement(query.getValue())) {

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlThreePidProvider.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class GenericSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public GenericSqlThreePidProvider(GenericSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/sql/SqlThreePidProvider.java

@@ -22,7 +22,7 @@ package io.kamax.mxisd.backend.sql;
 
 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.SqlConfig;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
@@ -30,8 +30,6 @@ import io.kamax.mxisd.lookup.provider.IThreePidProvider;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;
 
 import java.sql.Connection;
 import java.sql.PreparedStatement;
@@ -41,18 +39,16 @@ import java.util.ArrayList;
 import java.util.List;
 import java.util.Optional;
 
-@Component
+public abstract class SqlThreePidProvider implements IThreePidProvider {
-public class SqlThreePidProvider implements IThreePidProvider {
 
     private Logger log = LoggerFactory.getLogger(SqlThreePidProvider.class);
 
-    private SqlProviderConfig cfg;
+    private SqlConfig cfg;
     private MatrixConfig mxCfg;
 
     private SqlConnectionPool pool;
 
-    @Autowired
+    public SqlThreePidProvider(SqlConfig cfg, MatrixConfig mxCfg) {
-    public SqlThreePidProvider(SqlProviderConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.pool = new SqlConnectionPool(cfg);
         this.mxCfg = mxCfg;

src/main/java/io/kamax/mxisd/backend/sql/SynapseSqlThreePidProvider.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class SynapseSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public SynapseSqlThreePidProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/sql/SynapseSqliteDirectoryProvider.java

@@ -21,7 +21,7 @@
 package io.kamax.mxisd.backend.sql;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
 import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang.StringUtils;
@@ -32,9 +32,7 @@ import java.sql.PreparedStatement;
 import java.sql.SQLException;
 
 @Component
-public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
+public class SynapseSqliteDirectoryProvider extends GenericSqlDirectoryProvider {
-
-    private SynapseSqlProviderConfig cfg;
 
     @Autowired
     public SynapseSqliteDirectoryProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
@@ -42,7 +40,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
 
         if (StringUtils.equals("sqlite", cfg.getType())) {
             String userId = "'@' || p.user_id || ':" + mxCfg.getDomain() + "'";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
             queries.getName().setValue(
                     "select " + userId + ", displayname from profiles p where displayname like ?");
             queries.getThreepid().setValue(
@@ -51,7 +49,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
                             "where t.address like ?");
         } else if (StringUtils.equals("postgresql", cfg.getType())) {
             String userId = "concat('@',p.user_id,':" + mxCfg.getDomain() + "')";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
             queries.getName().setValue(
                     "select " + userId + ", displayname from profiles p where displayname ilike ?");
             queries.getThreepid().setValue(

src/main/java/io/kamax/mxisd/config/DirectoryConfig.java

@@ -18,45 +18,42 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.config.ldap;
+package io.kamax.mxisd.config;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
-
 @Configuration
-@ConfigurationProperties(prefix = "ldap.attribute")
+@ConfigurationProperties("directory")
-public class LdapAttributeConfig {
+public class DirectoryConfig {
 
-    private LdapAttributeUidConfig uid;
+    private final transient Logger log = LoggerFactory.getLogger(DnsOverwriteConfig.class);
-    private String name;
+
-    private Map<String, List<String>> threepid = new HashMap<>();
+    public static class Exclude {
+
+        private boolean homeserver;
+
+        public boolean getHomeserver() {
+            return homeserver;
+        }
+
+        public Exclude setHomeserver(boolean homeserver) {
+            this.homeserver = homeserver;
+            return this;
+        }
 
-    public LdapAttributeUidConfig getUid() {
-        return uid;
     }
 
-    public void setUid(LdapAttributeUidConfig uid) {
+    private Exclude exclude = new Exclude();
-        this.uid = uid;
+
+    public Exclude getExclude() {
+        return exclude;
     }
 
-    public String getName() {
+    public void setExclude(Exclude exclude) {
-        return name;
+        this.exclude = exclude;
-    }
-
-    public void setName(String name) {
-        this.name = name;
-    }
-
-    public Map<String, List<String>> getThreepid() {
-        return threepid;
-    }
-
-    public void setThreepid(Map<String, List<String>> threepid) {
-        this.threepid = threepid;
     }
 
 }

src/main/java/io/kamax/mxisd/config/InvitationConfig.java

@@ -0,0 +1,76 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import io.kamax.mxisd.util.GsonUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import javax.annotation.PostConstruct;
+
+@Configuration
+@ConfigurationProperties("invite")
+public class InvitationConfig {
+
+    private final Logger log = LoggerFactory.getLogger(InvitationConfig.class);
+
+    public static class Resolution {
+
+        private boolean recursive;
+        private long timer;
+
+        public boolean isRecursive() {
+            return recursive;
+        }
+
+        public void setRecursive(boolean recursive) {
+            this.recursive = recursive;
+        }
+
+        public long getTimer() {
+            return timer;
+        }
+
+        public void setTimer(long timer) {
+            this.timer = timer;
+        }
+
+    }
+
+    private Resolution resolution;
+
+    public Resolution getResolution() {
+        return resolution;
+    }
+
+    public void setResolution(Resolution resolution) {
+        this.resolution = resolution;
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- Invite config ---");
+        log.info("Resolution: {}", GsonUtil.build().toJson(resolution));
+    }
+
+}

src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java

@@ -22,28 +22,148 @@ package io.kamax.mxisd.config.ldap;
 
 import com.google.gson.Gson;
 import io.kamax.matrix.ThreePidMedium;
-import io.kamax.mxisd.backend.ldap.LdapGenericBackend;
+import io.kamax.mxisd.backend.ldap.LdapBackend;
 import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
 import javax.annotation.PostConstruct;
-import java.util.ArrayList;
+import java.util.*;
-import java.util.List;
 
 @Configuration
 @ConfigurationProperties(prefix = "ldap")
 public class LdapConfig {
 
-    private Logger log = LoggerFactory.getLogger(LdapConfig.class);
+    public static class UID {
-    private static Gson gson = new Gson();
 
-    private boolean enabled;
+        private String type;
-    private String filter;
+        private String value;
+
+        public String getType() {
+            return type;
+        }
+
+        public void setType(String type) {
+            this.type = type;
+        }
+
+        public String getValue() {
+            return value;
+        }
+
+        public void setValue(String value) {
+            this.value = value;
+        }
+
+    }
+
+    public static class Attribute {
+
+        private UID uid;
+        private String name;
+        private Map<String, List<String>> threepid = new HashMap<>();
+
+        public UID getUid() {
+            return uid;
+        }
+
+        public void setUid(UID uid) {
+            this.uid = uid;
+        }
+
+        public String getName() {
+            return name;
+        }
+
+        public void setName(String name) {
+            this.name = name;
+        }
+
+        public Map<String, List<String>> getThreepid() {
+            return threepid;
+        }
+
+        public void setThreepid(Map<String, List<String>> threepid) {
+            this.threepid = threepid;
+        }
+
+    }
+
+    public static class Auth {
+
+        private String filter;
+
+        public String getFilter() {
+            return filter;
+        }
+
+        public void setFilter(String filter) {
+            this.filter = filter;
+        }
+
+    }
+
+    public static class Connection {
+
+        private boolean tls;
+        private String host;
+        private int port;
+        private String bindDn;
+        private String bindPassword;
+        private String baseDn;
+
+        public boolean isTls() {
+            return tls;
+        }
+
+        public void setTls(boolean tls) {
+            this.tls = tls;
+        }
+
+        public String getHost() {
+            return host;
+        }
+
+        public void setHost(String host) {
+            this.host = host;
+        }
+
+        public int getPort() {
+            return port;
+        }
+
+        public void setPort(int port) {
+            this.port = port;
+        }
+
+        public String getBindDn() {
+            return bindDn;
+        }
+
+        public void setBindDn(String bindDn) {
+            this.bindDn = bindDn;
+        }
+
+        public String getBindPassword() {
+            return bindPassword;
+        }
+
+        public void setBindPassword(String bindPassword) {
+            this.bindPassword = bindPassword;
+        }
+
+        public String getBaseDn() {
+            return baseDn;
+        }
+
+        public void setBaseDn(String baseDn) {
+            this.baseDn = baseDn;
+        }
+
+    }
 
     public static class Directory {
 
@@ -82,12 +202,54 @@ public class LdapConfig {
 
     }
 
-    @Autowired
+    public static class Identity {
-    private LdapConnectionConfig conn;
+
-    private LdapAttributeConfig attribute;
+        private String filter;
-    private LdapAuthConfig auth;
+        private String token;
+        private Map<String, String> medium = new HashMap<>();
+
+        public String getFilter() {
+            return filter;
+        }
+
+        public void setFilter(String filter) {
+            this.filter = filter;
+        }
+
+        public String getToken() {
+            return token;
+        }
+
+        public void setToken(String token) {
+            this.token = token;
+        }
+
+        public Map<String, String> getMedium() {
+            return medium;
+        }
+
+        public Optional<String> getQuery(String key) {
+            return Optional.ofNullable(medium.get(key));
+        }
+
+        public void setMedium(Map<String, String> medium) {
+            this.medium = medium;
+        }
+
+    }
+
+
+    private Logger log = LoggerFactory.getLogger(LdapConfig.class);
+    private static Gson gson = new Gson();
+
+    private boolean enabled;
+    private String filter;
+
+    private Connection connection;
+    private Attribute attribute;
+    private Auth auth;
     private Directory directory;
-    private LdapIdentityConfig identity;
+    private Identity identity;
 
     public boolean isEnabled() {
         return enabled;
@@ -105,27 +267,27 @@ public class LdapConfig {
         this.filter = filter;
     }
 
-    public LdapConnectionConfig getConn() {
+    public Connection getConnection() {
-        return conn;
+        return connection;
     }
 
-    public void setConn(LdapConnectionConfig conn) {
+    public void setConnection(Connection conn) {
-        this.conn = conn;
+        this.connection = conn;
     }
 
-    public LdapAttributeConfig getAttribute() {
+    public Attribute getAttribute() {
         return attribute;
     }
 
-    public void setAttribute(LdapAttributeConfig attribute) {
+    public void setAttribute(Attribute attribute) {
         this.attribute = attribute;
     }
 
-    public LdapAuthConfig getAuth() {
+    public Auth getAuth() {
         return auth;
     }
 
-    public void setAuth(LdapAuthConfig auth) {
+    public void setAuth(Auth auth) {
         this.auth = auth;
     }
 
@@ -137,11 +299,11 @@ public class LdapConfig {
         this.directory = directory;
     }
 
-    public LdapIdentityConfig getIdentity() {
+    public Identity getIdentity() {
         return identity;
     }
 
-    public void setIdentity(LdapIdentityConfig identity) {
+    public void setIdentity(Identity identity) {
         this.identity = identity;
     }
 
@@ -154,25 +316,28 @@ public class LdapConfig {
             return;
         }
 
-        if (StringUtils.isBlank(conn.getHost())) {
+        if (StringUtils.isBlank(connection.getHost())) {
             throw new IllegalStateException("LDAP Host must be configured!");
         }
 
-        if (conn.getPort() < 1 || conn.getPort() > 65535) {
+        if (connection.getPort() < 1 || connection.getPort() > 65535) {
             throw new IllegalStateException("LDAP port is not valid");
         }
 
+        if (StringUtils.isBlank(connection.getBaseDn())) {
+            throw new ConfigurationException("ldap.connection.baseDn");
+        }
+
         if (StringUtils.isBlank(attribute.getUid().getType())) {
             throw new IllegalStateException("Attribute UID Type cannot be empty");
         }
 
-
         if (StringUtils.isBlank(attribute.getUid().getValue())) {
             throw new IllegalStateException("Attribute UID value cannot be empty");
         }
 
         String uidType = attribute.getUid().getType();
-        if (!StringUtils.equals(LdapGenericBackend.UID, uidType) && !StringUtils.equals(LdapGenericBackend.MATRIX_ID, uidType)) {
+        if (!StringUtils.equals(LdapBackend.UID, uidType) && !StringUtils.equals(LdapBackend.MATRIX_ID, uidType)) {
             throw new IllegalArgumentException("Unsupported LDAP UID type: " + uidType);
         }
 
@@ -184,9 +349,9 @@ public class LdapConfig {
         attribute.getThreepid().forEach((k, v) -> {
             if (StringUtils.isBlank(identity.getMedium().get(k))) {
                 if (ThreePidMedium.PhoneNumber.is(k)) {
-                    identity.getMedium().put(k, LdapGenericBackend.buildOrQuery("+" + getIdentity().getToken(), v));
+                    identity.getMedium().put(k, LdapBackend.buildOrQuery("+" + getIdentity().getToken(), v));
                 } else {
-                    identity.getMedium().put(k, LdapGenericBackend.buildOrQuery(getIdentity().getToken(), v));
+                    identity.getMedium().put(k, LdapBackend.buildOrQuery(getIdentity().getToken(), v));
                 }
             }
         });
@@ -195,10 +360,10 @@ public class LdapConfig {
         getDirectory().setFilter(StringUtils.defaultIfBlank(getDirectory().getFilter(), getFilter()));
         getIdentity().setFilter(StringUtils.defaultIfBlank(getIdentity().getFilter(), getFilter()));
 
-        log.info("Host: {}", conn.getHost());
+        log.info("Host: {}", connection.getHost());
-        log.info("Port: {}", conn.getPort());
+        log.info("Port: {}", connection.getPort());
-        log.info("Bind DN: {}", conn.getBindDn());
+        log.info("Bind DN: {}", connection.getBindDn());
-        log.info("Base DN: {}", conn.getBaseDn());
+        log.info("Base DN: {}", connection.getBaseDn());
 
         log.info("Attribute: {}", gson.toJson(attribute));
         log.info("Auth: {}", gson.toJson(auth));

src/main/java/io/kamax/mxisd/config/ldap/LdapConnectionConfig.java

@@ -1,85 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.ldap;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-@Configuration
-@ConfigurationProperties(prefix = "ldap.connection")
-public class LdapConnectionConfig {
-
-    private boolean tls;
-    private String host;
-    private int port;
-    private String bindDn;
-    private String bindPassword;
-    private String baseDn;
-
-    public boolean isTls() {
-        return tls;
-    }
-
-    public void setTls(boolean tls) {
-        this.tls = tls;
-    }
-
-    public String getHost() {
-        return host;
-    }
-
-    public void setHost(String host) {
-        this.host = host;
-    }
-
-    public int getPort() {
-        return port;
-    }
-
-    public void setPort(int port) {
-        this.port = port;
-    }
-
-    public String getBindDn() {
-        return bindDn;
-    }
-
-    public void setBindDn(String bindDn) {
-        this.bindDn = bindDn;
-    }
-
-    public String getBindPassword() {
-        return bindPassword;
-    }
-
-    public void setBindPassword(String bindPassword) {
-        this.bindPassword = bindPassword;
-    }
-
-    public String getBaseDn() {
-        return baseDn;
-    }
-
-    public void setBaseDn(String baseDn) {
-        this.baseDn = baseDn;
-    }
-
-}

src/main/java/io/kamax/mxisd/config/ldap/LdapIdentityConfig.java

@@ -1,66 +0,0 @@
-/*
- * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
- *
- * https://max.kamax.io/
- *
- * This program is free software: you can redistribute it and/or modify
- * it under the terms of the GNU Affero General Public License as
- * published by the Free Software Foundation, either version 3 of the
- * License, or (at your option) any later version.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU Affero General Public License for more details.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program.  If not, see <http://www.gnu.org/licenses/>.
- */
-
-package io.kamax.mxisd.config.ldap;
-
-import org.springframework.boot.context.properties.ConfigurationProperties;
-import org.springframework.context.annotation.Configuration;
-
-import java.util.HashMap;
-import java.util.Map;
-import java.util.Optional;
-
-@Configuration
-@ConfigurationProperties(prefix = "ldap.identity")
-public class LdapIdentityConfig {
-
-    private String filter;
-    private String token;
-    private Map<String, String> medium = new HashMap<>();
-
-    public String getFilter() {
-        return filter;
-    }
-
-    public void setFilter(String filter) {
-        this.filter = filter;
-    }
-
-    public String getToken() {
-        return token;
-    }
-
-    public void setToken(String token) {
-        this.token = token;
-    }
-
-    public Map<String, String> getMedium() {
-        return medium;
-    }
-
-    public Optional<String> getQuery(String key) {
-        return Optional.ofNullable(medium.get(key));
-    }
-
-    public void setMedium(Map<String, String> medium) {
-        this.medium = medium;
-    }
-
-}

src/main/java/io/kamax/mxisd/config/ldap/generic/GenericLdapConfig.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sàrl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,23 +18,16 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.config.ldap;
+package io.kamax.mxisd.config.ldap.generic;
 
+import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
+import org.springframework.context.annotation.Primary;
 
 @Configuration
-@ConfigurationProperties(prefix = "ldap.auth")
+@ConfigurationProperties(prefix = "ldap")
-public class LdapAuthConfig {
+@Primary
-
+public class GenericLdapConfig extends LdapConfig {
-    private String filter;
-
-    public String getFilter() {
-        return filter;
-    }
-
-    public void setFilter(String filter) {
-        this.filter = filter;
-    }
 
 }

src/main/java/io/kamax/mxisd/config/ldap/netiq/NetIqLdapConfig.java

@@ -1,8 +1,8 @@
 /*
  * mxisd - Matrix Identity Server Daemon
- * Copyright (C) 2017 Maxime Dor
+ * Copyright (C) 2018 Kamax Sàrl
  *
- * https://max.kamax.io/
+ * https://www.kamax.io/
  *
  * This program is free software: you can redistribute it and/or modify
  * it under the terms of the GNU Affero General Public License as
@@ -18,32 +18,14 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.config.ldap;
+package io.kamax.mxisd.config.ldap.netiq;
 
+import io.kamax.mxisd.config.ldap.generic.GenericLdapConfig;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
 @Configuration
-@ConfigurationProperties(prefix = "ldap.attribute.uid")
+@ConfigurationProperties(prefix = "netiq")
-public class LdapAttributeUidConfig {
+public class NetIqLdapConfig extends GenericLdapConfig {
-
-    private String type;
-    private String value;
-
-    public String getType() {
-        return type;
-    }
-
-    public void setType(String type) {
-        this.type = type;
-    }
-
-    public String getValue() {
-        return value;
-    }
-
-    public void setValue(String value) {
-        this.value = value;
-    }
 
 }

src/main/java/io/kamax/mxisd/config/memory/MemoryIdentityConfig.java

@@ -0,0 +1,59 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config.memory;
+
+import org.springframework.stereotype.Component;
+
+import java.util.ArrayList;
+import java.util.List;
+
+@Component
+public class MemoryIdentityConfig {
+
+    private String username;
+    private String password;
+    private List<MemoryThreePid> threepids = new ArrayList<>();
+
+    public String getUsername() {
+        return username;
+    }
+
+    public void setUsername(String username) {
+        this.username = username;
+    }
+
+    public String getPassword() {
+        return password;
+    }
+
+    public void setPassword(String password) {
+        this.password = password;
+    }
+
+    public List<MemoryThreePid> getThreepids() {
+        return threepids;
+    }
+
+    public void setThreepids(List<MemoryThreePid> threepids) {
+        this.threepids = threepids;
+    }
+
+}

src/main/java/io/kamax/mxisd/config/memory/MemoryStoreConfig.java

@@ -0,0 +1,51 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config.memory;
+
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import java.util.List;
+
+@Configuration
+@ConfigurationProperties("memory")
+public class MemoryStoreConfig {
+
+    private boolean enabled;
+    private List<MemoryIdentityConfig> identities;
+
+    public boolean isEnabled() {
+        return enabled;
+    }
+
+    public void setEnabled(boolean enabled) {
+        this.enabled = enabled;
+    }
+
+    public List<MemoryIdentityConfig> getIdentities() {
+        return identities;
+    }
+
+    public void setIdentities(List<MemoryIdentityConfig> identities) {
+        this.identities = identities;
+    }
+
+}

src/main/java/io/kamax/mxisd/config/memory/MemoryThreePid.java

@@ -0,0 +1,47 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config.memory;
+
+import org.springframework.stereotype.Component;
+
+@Component
+public class MemoryThreePid {
+
+    private String medium;
+    private String address;
+
+    public String getMedium() {
+        return medium;
+    }
+
+    public void setMedium(String medium) {
+        this.medium = medium;
+    }
+
+    public String getAddress() {
+        return address;
+    }
+
+    public void setAddress(String address) {
+        this.address = address;
+    }
+
+}

src/main/java/io/kamax/mxisd/config/sql/GenericSqlProviderConfig.java

@@ -24,21 +24,14 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 import org.springframework.context.annotation.Primary;
 
-import javax.annotation.PostConstruct;
-
 @Configuration
 @ConfigurationProperties("sql")
 @Primary
-public class SqlProviderConfig extends SqlConfig {
+public class GenericSqlProviderConfig extends SqlConfig {
 
     @Override
     protected String getProviderName() {
         return "Generic SQL";
     }
 
-    @PostConstruct
+}
-    public void build() {
-        super.build();
-    }
-
-}

src/main/java/io/kamax/mxisd/config/sql/SqlConfig.java

@@ -4,6 +4,7 @@ import io.kamax.mxisd.util.GsonUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.annotation.PostConstruct;
 import java.util.HashMap;
 import java.util.Map;
 
@@ -36,22 +37,22 @@ public abstract class SqlConfig {
 
     public static class Type {
 
-        private SqlProviderConfig.Query name = new SqlProviderConfig.Query();
+        private GenericSqlProviderConfig.Query name = new GenericSqlProviderConfig.Query();
-        private SqlProviderConfig.Query threepid = new SqlProviderConfig.Query();
+        private GenericSqlProviderConfig.Query threepid = new GenericSqlProviderConfig.Query();
 
-        public SqlProviderConfig.Query getName() {
+        public GenericSqlProviderConfig.Query getName() {
             return name;
         }
 
-        public void setName(SqlProviderConfig.Query name) {
+        public void setName(GenericSqlProviderConfig.Query name) {
             this.name = name;
         }
 
-        public SqlProviderConfig.Query getThreepid() {
+        public GenericSqlProviderConfig.Query getThreepid() {
             return threepid;
         }
 
-        public void setThreepid(SqlProviderConfig.Query threepid) {
+        public void setThreepid(GenericSqlProviderConfig.Query threepid) {
             this.threepid = threepid;
         }
 
@@ -74,7 +75,7 @@ public abstract class SqlConfig {
     public static class Directory {
 
         private Boolean enabled;
-        private SqlProviderConfig.Type query = new SqlProviderConfig.Type();
+        private GenericSqlProviderConfig.Type query = new GenericSqlProviderConfig.Type();
 
         public Boolean isEnabled() {
             return enabled;
@@ -84,11 +85,11 @@ public abstract class SqlConfig {
             this.enabled = enabled;
         }
 
-        public SqlProviderConfig.Type getQuery() {
+        public GenericSqlProviderConfig.Type getQuery() {
             return query;
         }
 
-        public void setQuery(SqlProviderConfig.Type query) {
+        public void setQuery(GenericSqlProviderConfig.Type query) {
             this.query = query;
         }
 
@@ -138,9 +139,9 @@ public abstract class SqlConfig {
     private boolean enabled;
     private String type;
     private String connection;
-    private SqlProviderConfig.Auth auth = new SqlProviderConfig.Auth();
+    private GenericSqlProviderConfig.Auth auth = new GenericSqlProviderConfig.Auth();
-    private SqlProviderConfig.Directory directory = new SqlProviderConfig.Directory();
+    private GenericSqlProviderConfig.Directory directory = new GenericSqlProviderConfig.Directory();
-    private SqlProviderConfig.Identity identity = new SqlProviderConfig.Identity();
+    private GenericSqlProviderConfig.Identity identity = new GenericSqlProviderConfig.Identity();
 
     public boolean isEnabled() {
         return enabled;
@@ -166,35 +167,33 @@ public abstract class SqlConfig {
         this.connection = connection;
     }
 
-    public SqlProviderConfig.Auth getAuth() {
+    public GenericSqlProviderConfig.Auth getAuth() {
         return auth;
     }
 
-    public void setAuth(SqlProviderConfig.Auth auth) {
+    public void setAuth(GenericSqlProviderConfig.Auth auth) {
         this.auth = auth;
     }
 
-    public SqlProviderConfig.Directory getDirectory() {
+    public GenericSqlProviderConfig.Directory getDirectory() {
         return directory;
     }
 
-    public void setDirectory(SqlProviderConfig.Directory directory) {
+    public void setDirectory(GenericSqlProviderConfig.Directory directory) {
         this.directory = directory;
     }
 
-    public SqlProviderConfig.Identity getIdentity() {
+    public GenericSqlProviderConfig.Identity getIdentity() {
         return identity;
     }
 
-    public void setIdentity(SqlProviderConfig.Identity identity) {
+    public void setIdentity(GenericSqlProviderConfig.Identity identity) {
         this.identity = identity;
     }
 
     protected abstract String getProviderName();
 
-    public void build() {
+    protected void doBuild() {
-        log.info("--- " + getProviderName() + " Provider config ---");
-
         if (getAuth().isEnabled() == null) {
             getAuth().setEnabled(isEnabled());
         }
@@ -206,6 +205,13 @@ public abstract class SqlConfig {
         if (getIdentity().isEnabled() == null) {
             getIdentity().setEnabled(isEnabled());
         }
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- " + getProviderName() + " Provider config ---");
+
+        doBuild();
 
         log.info("Enabled: {}", isEnabled());
         if (isEnabled()) {
@@ -214,6 +220,7 @@ public abstract class SqlConfig {
             log.info("Auth enabled: {}", getAuth().isEnabled());
             log.info("Directory queries: {}", GsonUtil.build().toJson(getDirectory().getQuery()));
             log.info("Identity type: {}", getIdentity().getType());
+            log.info("3PID mapping query: {}", getIdentity().getQuery());
             log.info("Identity medium queries: {}", GsonUtil.build().toJson(getIdentity().getMedium()));
         }
     }

src/main/java/io/kamax/mxisd/config/sql/synapse/SynapseSqlProviderConfig.java

@@ -21,6 +21,7 @@
 package io.kamax.mxisd.config.sql.synapse;
 
 import io.kamax.mxisd.config.sql.SqlConfig;
+import org.apache.commons.lang.StringUtils;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
@@ -36,8 +37,23 @@ public class SynapseSqlProviderConfig extends SqlConfig {
     }
 
     @PostConstruct
-    public void build() {
+    public void doBuild() {
-        super.build();
+        super.doBuild();
+        // FIXME check that the DB is not the mxisd one
+        // See https://matrix.to/#/!NPRUEisLjcaMtHIzDr:kamax.io/$1509377583327omXkC:kamax.io
+
+        getAuth().setEnabled(false); // Synapse does the auth, we only act as a directory/identity service.
+
+        if (getDirectory().isEnabled()) {
+            //FIXME set default queries for name and threepid
+        }
+
+        if (getIdentity().isEnabled()) {
+            if (StringUtils.isBlank(getIdentity().getType())) {
+                getIdentity().setType("mxid");
+                getIdentity().setQuery("SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?");
+            }
+        }
     }
 
 }

src/main/java/io/kamax/mxisd/controller/DefaultExceptionHandler.java

@@ -54,6 +54,16 @@ public class DefaultExceptionHandler {
         return gson.toJson(obj);
     }
 
+    @ExceptionHandler(RemoteLoginException.class)
+    public String handle(HttpServletRequest request, HttpServletResponse response, RemoteLoginException e) {
+        if (e.getErrorBodyMsgResp() != null) {
+            response.setStatus(e.getStatus());
+            log.info("Request {} {} - Error {}: {}", request.getMethod(), request.getRequestURL(), e.getErrorCode(), e.getError());
+            return gson.toJson(e.getErrorBodyMsgResp());
+        }
+        return handleGeneric(request, response, e);
+    }
+
     @ExceptionHandler(InternalServerError.class)
     public String handle(HttpServletRequest request, HttpServletResponse response, InternalServerError e) {
         if (StringUtils.isNotBlank(e.getInternalReason())) {

src/main/java/io/kamax/mxisd/controller/PingController.java

@@ -0,0 +1,38 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Maxime Dor
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.controller;
+
+import org.springframework.http.MediaType;
+import org.springframework.web.bind.annotation.CrossOrigin;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RestController;
+
+@RestController
+@CrossOrigin
+@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
+public class PingController {
+
+    @RequestMapping(value = "/_matrix/identity/api/v1")
+    public String ping() {
+        return "{}";
+    }
+
+}

src/main/java/io/kamax/mxisd/controller/auth/v1/AuthController.java

@@ -20,15 +20,27 @@
 
 package io.kamax.mxisd.controller.auth.v1;
 
-import com.google.gson.Gson;
+import com.google.gson.*;
-import com.google.gson.JsonElement;
+import com.google.i18n.phonenumbers.NumberParseException;
-import com.google.gson.JsonObject;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import com.google.i18n.phonenumbers.Phonenumber;
 import io.kamax.mxisd.auth.AuthManager;
 import io.kamax.mxisd.auth.UserAuthResult;
 import io.kamax.mxisd.controller.auth.v1.io.CredentialsValidationResponse;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
 import io.kamax.mxisd.exception.JsonMemberNotFoundException;
+import io.kamax.mxisd.exception.RemoteLoginException;
+import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.util.GsonParser;
 import io.kamax.mxisd.util.GsonUtil;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -39,13 +51,18 @@ import org.springframework.web.bind.annotation.RequestMethod;
 import org.springframework.web.bind.annotation.RestController;
 
 import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
 import java.io.IOException;
+import java.net.URI;
 
 @RestController
 @CrossOrigin
 @RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
 public class AuthController {
 
+    // TODO export into SDK
+    private static final String logV1Url = "/_matrix/client/r0/login";
+
     private Logger log = LoggerFactory.getLogger(AuthController.class);
 
     private Gson gson = GsonUtil.build();
@@ -54,6 +71,23 @@ public class AuthController {
     @Autowired
     private AuthManager mgr;
 
+    @Autowired
+    private LookupStrategy strategy;
+
+    @Autowired
+    private ClientDnsOverwrite dns;
+
+    @Autowired
+    private CloseableHttpClient client;
+
+    private String resolveProxyUrl(HttpServletRequest req) {
+        URI target = URI.create(req.getRequestURL().toString());
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
+
     @RequestMapping(value = "/_matrix-internal/identity/v1/check_credentials", method = RequestMethod.POST)
     public String checkCredentials(HttpServletRequest req) {
         try {
@@ -84,4 +118,112 @@ public class AuthController {
         }
     }
 
+    @RequestMapping(value = logV1Url, method = RequestMethod.GET)
+    public String getLogin(HttpServletRequest req, HttpServletResponse res) {
+        try (CloseableHttpResponse hsResponse = client.execute(new HttpGet(resolveProxyUrl(req)))) {
+            res.setStatus(hsResponse.getStatusLine().getStatusCode());
+            return EntityUtils.toString(hsResponse.getEntity());
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
+    @RequestMapping(value = logV1Url, method = RequestMethod.POST)
+    public String login(HttpServletRequest req) {
+        try {
+            JsonObject reqJsonObject = parser.parse(req.getInputStream());
+
+            // find 3PID in main object
+            GsonUtil.findPrimitive(reqJsonObject, "medium").ifPresent(medium -> {
+                GsonUtil.findPrimitive(reqJsonObject, "address").ifPresent(address -> {
+                    log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
+                    strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
+                        reqJsonObject.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
+                        reqJsonObject.remove("medium");
+                        reqJsonObject.remove("address");
+                    });
+                });
+            });
+
+            // find 3PID in 'identifier' object
+            GsonUtil.findObj(reqJsonObject, "identifier").ifPresent(identifier -> {
+                GsonUtil.findPrimitive(identifier, "type").ifPresent(type -> {
+
+                    if (StringUtils.equals(type.getAsString(), "m.id.thirdparty")) {
+                        GsonUtil.findPrimitive(identifier, "medium").ifPresent(medium -> {
+                            GsonUtil.findPrimitive(identifier, "address").ifPresent(address -> {
+                                log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
+                                strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
+                                    identifier.addProperty("type", "m.id.user");
+                                    identifier.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
+                                    identifier.remove("medium");
+                                    identifier.remove("address");
+                                });
+                            });
+                        });
+                    }
+
+                    if (StringUtils.equals(type.getAsString(), "m.id.phone")) {
+                        GsonUtil.findPrimitive(identifier, "number").ifPresent(number -> {
+                            GsonUtil.findPrimitive(identifier, "country").ifPresent(country -> {
+                                log.info("Login request with phone '{}'-'{}'", country.getAsString(), number.getAsString());
+                                try {
+                                    PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+                                    Phonenumber.PhoneNumber phoneNumber = phoneUtil.parse(number.getAsString(), country.getAsString());
+                                    String canon_phoneNumber = phoneUtil.format(phoneNumber, PhoneNumberUtil.PhoneNumberFormat.E164).replace("+", "");
+                                    String medium = "msisdn";
+                                    strategy.findLocal(medium, canon_phoneNumber).ifPresent(lookupDataOpt -> {
+                                        identifier.addProperty("type", "m.id.user");
+                                        identifier.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
+                                        identifier.remove("country");
+                                        identifier.remove("number");
+                                    });
+                                } catch (NumberParseException e) {
+                                    throw new RuntimeException(e);
+                                }
+                            });
+                        });
+                    }
+                });
+            });
+
+            // invoke 'login' on homeserver
+            HttpPost httpPost = RestClientUtils.post(resolveProxyUrl(req), gson, reqJsonObject);
+            try (CloseableHttpResponse httpResponse = client.execute(httpPost)) {
+                // check http status
+                int status = httpResponse.getStatusLine().getStatusCode();
+                log.info("http status = {}", status);
+                if (status != 200) {
+                    // try to get possible json error message from response
+                    // otherwise just get returned plain error message
+                    String errcode = String.valueOf(httpResponse.getStatusLine().getStatusCode());
+                    String error = EntityUtils.toString(httpResponse.getEntity());
+                    if (httpResponse.getEntity() != null) {
+                        try {
+                            JsonObject bodyJson = new JsonParser().parse(error).getAsJsonObject();
+                            if (bodyJson.has("errcode")) {
+                                errcode = bodyJson.get("errcode").getAsString();
+                            }
+                            if (bodyJson.has("error")) {
+                                error = bodyJson.get("error").getAsString();
+                            }
+                            throw new RemoteLoginException(status, errcode, error, bodyJson);
+                        } catch (JsonSyntaxException e) {
+                            log.warn("Response body is not JSON");
+                        }
+                    }
+                    throw new RemoteLoginException(status, errcode, error);
+                }
+
+                /// return response
+                JsonObject respJsonObject = parser.parseOptional(httpResponse).get();
+                return gson.toJson(respJsonObject);
+            } catch (IOException e) {
+                throw new RuntimeException(e);
+            }
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
 }

src/main/java/io/kamax/mxisd/controller/identity/v1/SessionRestController.java

@@ -104,12 +104,19 @@ public class SessionRestController {
 
         if (ThreePidMedium.PhoneNumber.is(medium)) {
             SessionPhoneTokenRequestJson req = parser.parse(request, SessionPhoneTokenRequestJson.class);
-            return gson.toJson(new Sid(mgr.create(
+            ThreePid threepid = new ThreePid(req.getMedium(), req.getValue());
+
+            String sessionId = mgr.create(
                     request.getRemoteHost(),
-                    new ThreePid(req.getMedium(), req.getValue()),
+                    threepid,
                     req.getSecret(),
                     req.getAttempt(),
-                    req.getNextLink())));
+                    req.getNextLink());
+
+            JsonObject res = new JsonObject();
+            res.addProperty("sid", sessionId);
+            res.addProperty(threepid.getMedium(), threepid.getAddress());
+            return gson.toJson(res);
         }
 
         JsonObject obj = new JsonObject();

src/main/java/io/kamax/mxisd/directory/DirectoryManager.java

@@ -23,6 +23,7 @@ package io.kamax.mxisd.directory;
 import com.google.gson.Gson;
 import com.google.gson.JsonSyntaxException;
 import io.kamax.matrix.MatrixErrorInfo;
+import io.kamax.mxisd.config.DirectoryConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchRequest;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.dns.ClientDnsOverwrite;
@@ -37,7 +38,6 @@ import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.utils.URIBuilder;
 import org.apache.http.entity.ContentType;
 import org.apache.http.impl.client.CloseableHttpClient;
-import org.apache.http.impl.client.HttpClients;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -54,16 +54,19 @@ public class DirectoryManager {
 
     private Logger log = LoggerFactory.getLogger(DirectoryManager.class);
 
+    private DirectoryConfig cfg;
     private List<IDirectoryProvider> providers;
 
     private ClientDnsOverwrite dns;
-    private CloseableHttpClient client;
     private Gson gson;
 
     @Autowired
-    public DirectoryManager(List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+    private CloseableHttpClient client;
+
+    @Autowired
+    public DirectoryManager(DirectoryConfig cfg, List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+        this.cfg = cfg;
         this.dns = dns;
-        this.client = HttpClients.custom().setUserAgent("mxisd").build(); //FIXME centralize
         this.gson = GsonUtil.build();
         this.providers = providers.stream().filter(IDirectoryProvider::isEnabled).collect(Collectors.toList());
 
@@ -76,37 +79,41 @@ public class DirectoryManager {
         log.info("Original request URL: {}", target);
         UserDirectorySearchResult result = new UserDirectorySearchResult();
 
-        URIBuilder builder = dns.transform(target);
+        if (cfg.getExclude().getHomeserver()) {
-        log.info("Querying HS at {}", builder);
+            log.info("Skipping HS directory data, disabled in config");
-        builder.setParameter("access_token", accessToken);
+        } else {
-        HttpPost req = RestClientUtils.post(
+            URIBuilder builder = dns.transform(target);
-                builder.toString(),
+            log.info("Querying HS at {}", builder);
-                new UserDirectorySearchRequest(query));
+            builder.setParameter("access_token", accessToken);
-        try (CloseableHttpResponse res = client.execute(req)) {
+            HttpPost req = RestClientUtils.post(
-            int status = res.getStatusLine().getStatusCode();
+                    builder.toString(),
-            Charset charset = ContentType.getOrDefault(res.getEntity()).getCharset();
+                    new UserDirectorySearchRequest(query));
-            String body = IOUtils.toString(res.getEntity().getContent(), charset);
+            try (CloseableHttpResponse res = client.execute(req)) {
+                int status = res.getStatusLine().getStatusCode();
+                Charset charset = ContentType.getOrDefault(res.getEntity()).getCharset();
+                String body = IOUtils.toString(res.getEntity().getContent(), charset);
 
-            if (status != 200) {
+                if (status != 200) {
-                MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
+                    MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
-                if (StringUtils.equals("M_UNRECOGNIZED", info.getErrcode())) { // FIXME no hardcoding, use Enum
+                    if (StringUtils.equals("M_UNRECOGNIZED", info.getErrcode())) { // FIXME no hardcoding, use Enum
-                    log.warn("Homeserver does not support Directory feature, skipping");
+                        log.warn("Homeserver does not support Directory feature, skipping");
-                } else {
+                    } else {
-                    log.error("Homeserver returned an error while performing directory search");
+                        log.error("Homeserver returned an error while performing directory search");
-                    throw new MatrixException(status, info.getErrcode(), info.getError());
+                        throw new MatrixException(status, info.getErrcode(), info.getError());
+                    }
                 }
-            }
 
-            UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
+                UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
-            log.info("Found {} match(es) in HS for '{}'", resultHs.getResults().size(), query);
+                log.info("Found {} match(es) in HS for '{}'", resultHs.getResults().size(), query);
-            result.getResults().addAll(resultHs.getResults());
+                result.getResults().addAll(resultHs.getResults());
-            if (resultHs.isLimited()) {
+                if (resultHs.isLimited()) {
-                result.setLimited(true);
+                    result.setLimited(true);
+                }
+            } catch (JsonSyntaxException e) {
+                throw new InternalServerError("Invalid JSON reply from the HS: " + e.getMessage());
+            } catch (IOException e) {
+                throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
             }
-        } catch (JsonSyntaxException e) {
-            throw new InternalServerError("Invalid JSON reply from the HS: " + e.getMessage());
-        } catch (IOException e) {
-            throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
         }
 
         for (IDirectoryProvider provider : providers) {

src/main/java/io/kamax/mxisd/exception/RemoteLoginException.java

@@ -0,0 +1,43 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.exception;
+
+
+import com.google.gson.JsonObject;
+
+public class RemoteLoginException extends MatrixException {
+
+    private JsonObject errorBodyMsgResp;
+
+    public RemoteLoginException(int status, String errorCode, String error) {
+        super(status, errorCode, error);
+        this.errorBodyMsgResp = null;
+    }
+
+    public RemoteLoginException(int status, String errorCode, String error, JsonObject errorBodyMsgResp) {
+        super(status, errorCode, error);
+        this.errorBodyMsgResp = errorBodyMsgResp;
+    }
+
+    public JsonObject getErrorBodyMsgResp() {
+        return errorBodyMsgResp;
+    }
+}

src/main/java/io/kamax/mxisd/invitation/InvitationManager.java

@@ -24,6 +24,7 @@ import com.google.gson.Gson;
 import com.google.gson.JsonArray;
 import com.google.gson.JsonObject;
 import io.kamax.matrix.MatrixID;
+import io.kamax.mxisd.config.InvitationConfig;
 import io.kamax.mxisd.dns.FederationDnsOverwrite;
 import io.kamax.mxisd.exception.BadRequestException;
 import io.kamax.mxisd.exception.MappingAlreadyExistsException;
@@ -72,6 +73,9 @@ public class InvitationManager {
 
     private Map<String, IThreePidInviteReply> invitations = new ConcurrentHashMap<>();
 
+    @Autowired
+    private InvitationConfig cfg;
+
     @Autowired
     private IStorage storage;
 
@@ -137,7 +141,7 @@ public class InvitationManager {
                     log.error("Error when running background mapping refresh", t);
                 }
             }
-        }, 5000L, TimeUnit.MILLISECONDS.convert(1, TimeUnit.MINUTES)); // FIXME make configurable
+        }, 5000L, TimeUnit.MILLISECONDS.convert(cfg.getResolution().getTimer(), TimeUnit.MINUTES));
     }
 
     @PreDestroy
@@ -204,6 +208,14 @@ public class InvitationManager {
         return "https://" + domain + ":8448";
     }
 
+    private Optional<SingleLookupReply> lookup3pid(String medium, String address) {
+        if (!cfg.getResolution().isRecursive()) {
+            log.warn("/!\\ /!\\ --- RECURSIVE INVITE RESOLUTION HAS BEEN DISABLED --- /!\\ /!\\");
+        }
+
+        return lookupMgr.find(medium, address, cfg.getResolution().isRecursive());
+    }
+
     public synchronized IThreePidInviteReply storeInvite(IThreePidInvite invitation) { // TODO better sync
         if (!notifMgr.isMediumSupported(invitation.getMedium())) {
             throw new BadRequestException("Medium type " + invitation.getMedium() + " is not supported");
@@ -223,7 +235,7 @@ public class InvitationManager {
             return reply;
         }
 
-        Optional<?> result = lookupMgr.find(invitation.getMedium(), invitation.getAddress(), true);
+        Optional<SingleLookupReply> result = lookup3pid(invitation.getMedium(), invitation.getAddress());
         if (result.isPresent()) {
             log.info("Mapping for {}:{} already exists, refusing to store invite", invitation.getMedium(), invitation.getAddress());
             throw new MappingAlreadyExistsException();
@@ -333,7 +345,7 @@ public class InvitationManager {
         public void run() {
             try {
                 log.info("Searching for mapping created since invite {} was created", getIdForLog(reply));
-                Optional<SingleLookupReply> result = lookupMgr.find(reply.getInvite().getMedium(), reply.getInvite().getAddress(), true);
+                Optional<SingleLookupReply> result = lookup3pid(reply.getInvite().getMedium(), reply.getInvite().getAddress());
                 if (result.isPresent()) {
                     SingleLookupReply lookup = result.get();
                     log.info("Found mapping for pending invite {}", getIdForLog(reply));

src/main/java/io/kamax/mxisd/lookup/strategy/RecursivePriorityLookupStrategy.java

@@ -53,7 +53,10 @@ public class RecursivePriorityLookupStrategy implements LookupStrategy {
     public RecursivePriorityLookupStrategy(RecursiveLookupConfig cfg, List<IThreePidProvider> providers, IBridgeFetcher bridge) {
         this.cfg = cfg;
         this.bridge = bridge;
-        this.providers = providers.stream().filter(IThreePidProvider::isEnabled).collect(Collectors.toList());
+        this.providers = providers.stream().filter(p -> {
+            log.info("3PID Provider {} is enabled: {}", p.getClass().getSimpleName(), p.isEnabled());
+            return p.isEnabled();
+        }).collect(Collectors.toList());
     }
 
     @PostConstruct
@@ -156,6 +159,8 @@ public class RecursivePriorityLookupStrategy implements LookupStrategy {
         for (IThreePidProvider provider : providers) {
             Optional<SingleLookupReply> lookupDataOpt = provider.find(request);
             if (lookupDataOpt.isPresent()) {
+                log.info("Found 3PID mapping: {medium: '{}', address: '{}', mxid: '{}'}",
+                        request.getType(), request.getThreePid(), lookupDataOpt.get().getMxid().getId());
                 return lookupDataOpt;
             }
         }
@@ -166,9 +171,13 @@ public class RecursivePriorityLookupStrategy implements LookupStrategy {
                         (!cfg.getBridge().getRecursiveOnly() || isAllowedForRecursive(request.getRequester()))
                 ) {
             log.info("Using bridge failover for lookup");
-            return bridge.find(request);
+            Optional<SingleLookupReply> lookupDataOpt = bridge.find(request);
+            log.info("Found 3PID mapping: {medium: '{}', address: '{}', mxid: '{}'}",
+                    request.getThreePid(), request.getId(), lookupDataOpt.get().getMxid().getId());
+            return lookupDataOpt;
         }
 
+        log.info("No 3PID mapping found");
         return Optional.empty();
     }
 

src/main/java/io/kamax/mxisd/notification/NotificationManager.java

@@ -46,16 +46,14 @@ public class NotificationManager {
         this.handlers = new HashMap<>();
         handlers.forEach(h -> {
             log.info("Found handler {} for medium {}", h.getId(), h.getMedium());
-            String handlerId = cfg.getHandler().get(h.getMedium());
+            String handlerId = cfg.getHandler().getOrDefault(h.getMedium(), "raw");
-            if (StringUtils.isBlank(handlerId) || StringUtils.equals(handlerId, h.getId())) {
+            if (StringUtils.equals(handlerId, h.getId())) {
                 this.handlers.put(h.getMedium(), h);
             }
         });
 
         log.info("--- Notification handler ---");
-        this.handlers.forEach((k, v) -> {
+        this.handlers.forEach((k, v) -> log.info("\tHandler for {}: {}", k, v.getId()));
-            log.info("\tHandler for {}: {}", k, v.getId());
-        });
     }
 
     private INotificationHandler ensureMedium(String medium) {

src/main/java/io/kamax/mxisd/session/SessionMananger.java

@@ -196,7 +196,7 @@ public class SessionMananger {
         storage.updateThreePidSession(session.getDao());
         log.info("Session {} has been validated locally", session.getId());
 
-        if (ThreePidMedium.PhoneNumber.is(session.getThreePid().getMedium()) && session.isValidated()) {
+        if (ThreePidMedium.PhoneNumber.is(session.getThreePid().getMedium()) && session.isValidated() && policy.toRemote()) {
             createRemote(sid, secret);
             // FIXME make the message configurable/customizable (templates?)
             throw new MessageForClientException("You will receive a NEW code from another number. Enter it below");
@@ -379,7 +379,7 @@ public class SessionMananger {
 
             if (o.has("validated_at")) {
                 ThreePid remoteThreePid = new ThreePid(o.get("medium").getAsString(), o.get("address").getAsString());
-                if (session.getThreePid().equals(remoteThreePid)) { // sanity check
+                if (!session.getThreePid().equals(remoteThreePid)) { // sanity check
                     throw new InternalServerError("Local 3PID " + session.getThreePid() + " and remote 3PID " + remoteThreePid + " do not match for session " + session.getId());
                 }
 

src/main/java/io/kamax/mxisd/spring/CloseableHttpClientFactory.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.spring;
+
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.springframework.context.annotation.Bean;
+import org.springframework.context.annotation.Configuration;
+
+@Configuration
+public class CloseableHttpClientFactory {
+
+    @Bean
+    public CloseableHttpClient getClient() {
+        return HttpClients.custom().setUserAgent("mxisd").build();
+    }
+
+}

src/main/java/io/kamax/mxisd/threepid/connector/phone/BlackholePhoneConnector.java

@@ -0,0 +1,38 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sàrl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.threepid.connector.phone;
+
+import org.springframework.stereotype.Component;
+
+@Component
+public class BlackholePhoneConnector implements IPhoneConnector {
+
+    @Override
+    public void send(String recipient, String content) {
+        //dev/null
+    }
+
+    @Override
+    public String getId() {
+        return "BLACKHOLE";
+    }
+
+}

src/main/java/io/kamax/mxisd/util/GsonUtil.java

@@ -20,9 +20,9 @@
 
 package io.kamax.mxisd.util;
 
-import com.google.gson.FieldNamingPolicy;
+import com.google.gson.*;
-import com.google.gson.Gson;
+
-import com.google.gson.GsonBuilder;
+import java.util.Optional;
 
 public class GsonUtil {
 
@@ -30,4 +30,16 @@ public class GsonUtil {
         return new GsonBuilder().setFieldNamingPolicy(FieldNamingPolicy.LOWER_CASE_WITH_UNDERSCORES).create();
     }
 
+    public static Optional<JsonElement> findElement(JsonObject o, String key) {
+        return Optional.ofNullable(o.get(key));
+    }
+
+    public static Optional<JsonObject> findObj(JsonObject o, String key) {
+        return findElement(o, key).map(el -> el.isJsonObject() ? el.getAsJsonObject() : null);
+    }
+
+    public static Optional<JsonPrimitive> findPrimitive(JsonObject o, String key) {
+        return findElement(o, key).map(el -> el.isJsonPrimitive() ? el.getAsJsonPrimitive() : null);
+    }
+
 }

src/main/resources/application.yaml

@@ -89,6 +89,46 @@ ldap:
       email: ''
       msisdn: ''
 
+netiq:
+  enabled: false
+  filter: ''
+  connection:
+    host: ''
+    tls: false
+    port: 389
+    bindDn: ''
+    bindPassword: ''
+    baseDn: ''
+  attribute:
+    uid:
+      type: 'uid'
+      value: 'userPrincipalName'
+    name: 'displayName'
+    threepid:
+      email:
+        - 'mailPrimaryAddress'
+        - 'mail'
+        - 'otherMailbox'
+      msisdn:
+        - 'telephoneNumber'
+        - 'mobile'
+        - 'homePhone'
+        - 'otherTelephone'
+        - 'otherMobile'
+        - 'otherHomePhone'
+  auth:
+    filter: ''
+  directory:
+    attribute:
+      other: []
+    filter: ''
+  identity:
+    filter: ''
+    token: '%3pid'
+    medium:
+      email: ''
+      msisdn: ''
+
 firebase:
   enabled: false
 
@@ -224,13 +264,22 @@ view:
         success: 'session/remote/checkSuccess'
         failure: 'session/remote/checkFailure'
 
+invite:
+  resolution:
+    recursive: true
+    timer: 1
+
 storage:
   backend: 'sqlite'
 
+directory:
+  exclude:
+    homeserver: false
+
 ---
 spring:
   profiles: systemd
 
 logging:
   pattern:
-    console: '%d{.SSS}${LOG_LEVEL_PATTERN:%5p} [%15.15t] %35.35logger{34} : %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}'
+    console: '%d{.SSS} ${LOG_LEVEL_PATTERN:%5p} [%15.15t] %35.35logger{34} : %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}'