Improve the docker experience

- Only one env variable to configure on first usage - Auto-generate a default config - Improve doc
Take LDAP filter into account when doing 3PID lookups
2017-12-16 19:58:11 +01:00 · 2017-12-16 19:17:22 +01:00 · 2017-12-06 20:31:06 +01:00 · 2017-12-02 20:13:25 +01:00 · 2017-11-30 02:44:21 +01:00 · 2017-11-26 16:30:42 +01:00
40 changed files with 823 additions and 334 deletions
--- a/5
+++ b/5
@@ -6,6 +6,11 @@ EXPOSE 8090

 ADD build/libs/mxisd.jar /mxisd.jar
 ADD src/docker/start.sh /start.sh
+RUN mkdir -p /var/mxisd

 ENV JAVA_OPTS=""
+ENV CONF_FILE_PATH="/etc/mxisd/mxisd.yaml"
+ENV SIGN_KEY_PATH="/var/mxisd/sign.key"
+ENV SQLITE_DATABASE_PATH="/var/mxisd/mxisd.db"
+
 CMD [ "/start.sh" ]
--- a/README.md
+++ b/README.md
@@ -5,20 +5,20 @@ mxisd - Federated Matrix Identity Server Daemon
 - [Overview](#overview)
 - [Features](#features)
 - [Why use mxisd](#why-use-mxisd)
- [Quick start](#quick-start)
+- [Getting Started](#getting-started)
 - [Support](#support)
 - [Contribute](#contribute)
 - [FAQ](#faq)
 - [Contact](#contact)

 # Overview
-mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with enhanced features.
+mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with [enhanced features](#features).
  
 It is specifically designed to connect to an Identity store (AD/Samba/LDAP, SQL Database, Web services/application, ...)
 and ease the integration of the Matrix ecosystem with an existing infrastructure, or to build a new one using lasting
 tools.

-The core principle of mxisd is to map between Matrix IDs and 3PIDs (Thrid-party Identifiers) for the Homeserver and its
+The core principle of mxisd is to map between Matrix IDs and 3PIDs (Third-party Identifiers) for the Homeserver and its
 users. 3PIDs can be anything that identify a user, like:
 - Full name
 - Email address
@@ -29,8 +29,9 @@ users. 3PIDs can be anything that identify a user, like:
 - Facebook ID
 - ...

-mxisd is an enhanced Identity service, which implements the [Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html)
-but also several other features that greatly enhance user experience within Matrix.
+mxisd is an enhanced Identity service, which implements the
+[Matrix Identity service API](https://matrix.org/docs/spec/identity_service/unstable.html) but also several
+[other features](#features) that greatly enhance user experience within Matrix.

 mxisd is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built as a
 single coherent product.
@@ -64,114 +65,20 @@ currently **cannot be removed**
 - Users can directly find each other using whatever attribute is relevant within your Identity store
 - Federate your Identity lookups so you can discover others and/or others can discover you, all with extensive ACLs

-# Quick Start
-1. [Preparation](#preparation)
-2. [Install](#install)
-3. [Configure](#configure)
-4. [Integrate](#integrate)
-5. [Validate](#validate)
-
-Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
-talk to the central Matrix.org Identity service.  
-This will be a good ground work for further integration with your existing Identity stores.
-
-## Preparation
-You will need:
- Homeserver
- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
-
-As synapse requires an HTTPS connection when talking to an Identity service, a reverse proxy is required as mxisd does
-not support HTTPS listener at this time.
-
-For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.  
-You can also use a dedicated domain for mxisd, but will not have access to some features.
-
-Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-io/mxisd/wiki/Gotchas#nating) if you use the same
-hostname.
-
-The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
-If you would like a high-level view of the infrastructure and how each feature is integrated, see the
-[dedicated document](docs/architecture.md)
-
-## Install
-Install via:
- [Debian package](docs/install/debian.md)
- [Docker image](docs/install/docker.md)
- [Sources](docs/build.md)
-
-See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
-
-## Configure
-Create/edit a minimal configuration (see installer doc for the location):
-```
-matrix.domain: 'MyMatrixDomain.org'
-key.path: '/path/to/signing.key.file'
-storage.provider.sqlite.database: '/path/to/mxisd.db'
-```  
- `matrix.domain` should be set to your Homeserver domain
- `key.path` will store the signing keys, which must be kept safe!
- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
-
-If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
-Complete configuration guide is available [here](docs/configure.md).
-
-## Integrate
-For an overview of a typical mxisd infrastructure, see the [dedicated document](docs/architecture.md)
-### Reverse proxy
-#### Apache2
-In the VirtualHost handling the domain with SSL, add the following line and replace `0.0.0.0` by the right address/host.  
-**This line MUST be present before the one for the homeserver!**
-```
-ProxyPass /_matrix/identity http://0.0.0.0:8090/_matrix/identity
-```
-
-Typical VirtualHost configuration would be:
-```
-<VirtualHost *:443>
-    ServerName example.org
-    
-    ...
-    
-    ProxyPreserveHost on
-    ProxyPass /_matrix/identity/ http://10.1.2.3:8090/_matrix/identity/
-    ProxyPass /_matrix/ http://10.1.2.3:8008/_matrix/
-</VirtualHost>
-```
-
-### Synapse
-Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
-In a typical configuration, you would end up with something similair to:
-```
-trusted_third_party_id_servers:
-    - matrix.org
-    - vector.im
-    - example.org
-```
-It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is allowed by synapse. 
-
-### Federation and network discovery
-See the [dedicated document](docs/features/federation.md).
-
-## Validate
-Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
-the relevant hostname which you configured in your reverse proxy.
-Try to invite `mxisd-lookup-test@kamax.io`, which should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`
-  
-If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
-If it did not work, [get in touch](#support) and we'll do our best to get you started.  
-
-You can now integrate mxisd further with your infrastructure using the various [features](docs/README.md) guides.
+# Getting started
+See the [dedicated document](docs/getting-started.md)

 # Support
 ## Community
 If you need help, want to report a bug or just say hi, you can reach us on Matrix at 
-[#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) or [directly peek anonymously](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/).
+[#mxisd:kamax.io](https://matrix.to/#/#mxisd:kamax.io) or
+[directly peek anonymously](https://view.matrix.org/room/!NPRUEisLjcaMtHIzDr:kamax.io/).
 For more high-level discussion about the Identity Server architecture/API, go to 
 [#matrix-identity:matrix.org](https://matrix.to/#/#matrix-identity:matrix.org)

 ## Professional
-If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open source technologies/products, 
-please visit [our website](https://www.kamax.io/) to get in touch with us and get a quote.
+If you would prefer professional support/custom development for mxisd and/or for Matrix in general, including other open
+source technologies/products, please visit [our website](https://www.kamax.io/) to get in touch with us and get a quote.

 We offer affordable monthly/yearly support plans for mxisd, synapse or your full Matrix infrastructure.

@@ -184,8 +91,8 @@ You can contribute as a community member by:
 - Helping us improve the documentation: tell us what is good or not good (in an issue or in Matrix), or make a PR with
 changes you feel improve the doc.
 - Contribute code directly: we love contributors! All your contributions will be licensed under AGPLv3.
- Donate! any donation is welcome, regardless how small or big. This will directly be used for the fixed costs and
-developer time.
+- [Donate!](https://liberapay.com/maximusdor/) Any donation is welcome, regardless how small or big, and will directly
+be used for the fixed costs and developer time of mxisd.

 You can contribute as an organisation/corporation by:
 - Get a [support contract](#support-professional). This is the best way you can help us as it ensures mxisd is
@@ -193,64 +100,7 @@ maintained regularly and you get direct access to the support team.
 - Sponsoring new features or bug fixes. [Get in touch](#contact) so we can discuss it further.

 # FAQ
-### Do I need to use mxisd if I run a Homeserver?
-No, but it is recommended, even if you don't use any backends or integration.
-
-mxisd in its default configuration will use federation and involve the central Matrix.org Identity servers when
-performing queries, giving you access to at least the same information as if you were not running it.
-
-It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
-privacy consequences, which is not the case with the central Matrix.org servers.
-
-So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
-simple features on top of it.
-
-### I already use the synapse LDAP3 auth provider, why should I care about mxisd?
-The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) only handles on specific flow:
-validate credentials at login.
-
-It does not:
- Auto-provision user profiles
- Integrate with Identity management
- Integrate with Directory searches
- Protect you against the username case sensitivites issues in synapse
-
-mxisd is a replacement and enhancement of it, and also offers coherent results in all areas, which LDAP3 auth provider
-does not.
-
-### I saw that sydent is the official Identity server implemenation of the Matrix team, I should use that!
-You can, but [sydent](https://github.com/matrix-org/sydent):
- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
- is not meant to be linked to a specific Homeserver / domain
- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
- forces you to duplicate all your identity data, so people can be found by 3PIDs
- forces users to enter all their emails and phone numbers manually in their profile
-
-So really, you should go with mxisd.
-
-### I'm not sure I understand what an "Identity server" is supposed to be or do
-The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
-what they want to do with that part of the ecosystem. Therefore, "Identity" is a misleading word currently.  
-Given the scope of the current Identity Service API, it would be best called "Invitation service".
-
-Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
-for groups/corporations/organisation. This is where mxisd comes in.
-
-mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
-up to its "Identity server" name.
-
-### So mxisd is just a big hack! I don't want to use non-official features!
-mxisd primary concern is to always be compatible with the Matrix ecosystem and the Identity service API.  
-Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
-
-We also directly talk with the Matrix developers to ensure all features we implement have their approval, and that we
-are in line with their vision of Identity management within the Matrix ecosystem.
-
-Therefore, using mxisd is a safe choice. It will be like using the central Matrix.org Identity servers, yet not closing
-the door to very nice enhancements and integrations.
-
-### Should I use mxisd if I don't host my own Homeserver?
-No
+See the [dedicated document](docs/faq.md)

 # Contact
 Get in touch via:
--- a/build.gradle
+++ b/build.gradle
@@ -47,7 +47,7 @@ String gitVersion() {
    def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")
    ByteArrayOutputStream out = new ByteArrayOutputStream()
    exec {
-        commandLine = ['git', 'describe', '--always', '--dirty']
+        commandLine = ['git', 'describe', '--tags', '--always', '--dirty']
        standardOutput = out
    }
    def v = out.toString().replace(System.lineSeparator(), '')
--- a/docs/_config.yml
+++ b/docs/_config.yml
@@ -0,0 +1 @@
+theme: jekyll-theme-cayman
--- a/docs/backends/README.md
+++ b/docs/backends/README.md
@@ -0,0 +1,5 @@
+# Identity Stores (Backends)
+- [Samba / Active Directory / LDAP](ldap.md)
+- [SQL Databases](sql.md)
+- [Website / Web service / Web app](rest.md)
+- [Google Firebase](firebase.md)
--- a/docs/backends/firebase.md
+++ b/docs/backends/firebase.md
@@ -1,4 +1,14 @@
 # Google Firebase
+https://firebase.google.com/
+
+## Requirements
+This backend requires a suitable Matrix client capable of performing Firebase authentication and passing the following
+information:
+- Firebase User ID as Matrix username
+- Firebase token as Matrix password
+
+If your client is Riot, you will need a custom version.
+
 ## Configuration
 To be completed. For now, see default structure and values:
 ```
@@ -6,4 +16,4 @@ firebase:
  enabled: false
  credentials: '/path/to/firebase/credentials.json'
  database: 'https://my-project.firebaseio.com/'
-```
+```
--- a/docs/backends/ldap.md
+++ b/docs/backends/ldap.md
@@ -1,53 +1,99 @@
-# AD/Samba/LDAP backend
+# LDAP (Samba / Active Directory / OpenLDAP)
+## Getting started
+To use your LDAP backend, add the bare minimum configuration in mxisd config file:
+```
+ldap.enabled: true
+ldap.connection.host: 'ldapHostnameOrIp'
+ldap.connection.bindDn: 'CN=My Mxisd User,OU=Users,DC=example,DC=org'
+ldap.connection.bindPassword: 'TheUserPassword'
+ldap.connection.baseDn: 'OU=Users,DC=example,DC=org'
+```
+These are standard LDAP connection configuration. mxisd will try to connect on port default port 389 without encryption.
+
+---
+
+If you would like to use a TLS/SSL connection, use the following configuration options (STARTLS not supported):
+```
+ldap.connection.tls: true
+ldap.connection.port: 12345
+```
+
+---
+
+You can also set a default global filter on any LDAP queries:
+```
+ldap.filter: '(memberOf=CN=My Matrix Users,OU=Groups,DC=example,DC=org)'
+```
+This example would only return users part of the group called `My Matrix Users`.
+This can be overwritten or append in each specific flow describe below.
+
+---
+
+LDAP features are based on mapping LDAP attributes to Matrix concepts, like a Matrix ID, its localpart, the user display
+name, their email(s) and/or phone number(s).
+     
+Default attributes are well suited for Active Directory/Samba. In case you are using a native LDAP backend, you will
+most certainly configure those mappings.
+
+The following example would set the `uid` attribute as localpart and the Matrix display name to `cn`
+```
+ldap.attribute.uid.type: 'uid'
+ldap.attribute.uid.value: 'uid'
+ldap.attribute.name: 'cn'
+```
+
+You can also change the attribute lists for 3PID, like email or phone numbers.  
+The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67)
+for emails and phone number:
+```
+ldap.attribute.threepid.email:
+  - 'mail'
+  - 'otherMailAttribute'
+
+ldap.attribute.threepid.msisdn:
+  - 'phone'
+  - 'otherPhoneAttribute'
+```
+
+## Identity
+Identity features (related to 3PID invites or searches) are enabled and configured using default values and no specific
+configuration item is needed to get started.
+
+If you would like to overwrite some global configuration relative to filter and/or attributes, see the Identity section
+of the Configuration below.
+
+## Authentication
+No further configuration is needed to enable authentication with LDAP once globally enabled and configured.  
+You have the possiblity to use a different query filter if you wish, see Configuration below.
+
+Profile auto-fill is enabled by default. It will use the `name` and `threepid` configuration options to get a lit of
+attributes to be used to build the user profile to pass on to synapse during authentication.
+
+## Directory
+No further configuration is needed to enable directory with LDAP once globally enabled and configured.
+
+If you would like to use extra attributes in search that are not 3PIDs, like nicknames, group names, employee number:
+```
+ldap.directory.attribute.other:
+  - 'myNicknameAttribute'
+  - 'memberOf'
+  - 'employeeNumberAttribute'
+```
+
 ## Configuration
-### Structure and default values
-```
-ldap:
-  enabled: false
-  filter: ''
-  connection:
-    host: ''
-    tls: false
-    port: 389
-    bindDn: ''
-    bindPassword: ''
-    baseDn: ''
-  attribute:
-    uid:
-      type: 'uid'
-      value: 'userPrincipalName'
-    name: 'displayName'
-    threepid:
-      email:
-        - 'mailPrimaryAddress'
-        - 'mail'
-        - 'otherMailbox'
-      msisdn:
-        - 'telephoneNumber'
-        - 'mobile'
-        - 'homePhone'
-        - 'otherTelephone'
-        - 'otherMobile'
-        - 'otherHomePhone'
-  auth:
-    filter: ''
-  directory:
-    attribute:
-      other: []
-    filter: ''
-  identity:
-    filter: ''
-    medium:
-      email: ''
-      msisdn: ''
-```
+Please read the [Configuration](../configure.md) explanatory note if you are not familiar with the terms used below.
+ 
 ### General
+Base path: `ldap`
+
 | Item      | Description                                                                               |
 |-----------|-------------------------------------------------------------------------------------------|
 | `enabled` | Globaly enable/disable the LDAP backend                                                   |
 | `filter`  | Global filter to apply on all LDAP queries. Can be overwritten in each applicable section |

 ### Connection
+Base path: `ldap.connection`
+
 | Item           | Description                                          |
 |----------------|------------------------------------------------------|
 | `host`         | Host to connect to                                   |
@@ -58,6 +104,8 @@ ldap:
 | `baseDn`       | Base DN for queries                                  |

 ### Attributes
+Base path: `ldap.attribute`
+
 | Item        | Description                                                                                                            |
 |-------------|------------------------------------------------------------------------------------------------------------------------|
 | `uid.type`  | Indicate how to process the User ID (UID) attribute:                                                                   |
@@ -68,11 +116,15 @@ ldap:
 | `threepid`  | Namespace where each key is a 3PID type and contains a list of attributes                                              |

 ### Authentication
+Base path: `ldap.auth`
+
 | Item     | Description                                                                                      |
 |----------|--------------------------------------------------------------------------------------------------|
 | `filter` | Specific user filter applied during authentication. Global filter is used if empty/blank/not set |

 ### Directory
+Base path: `ldap.directory`
+
 | Item              | Description                                                         |
 |-------------------|---------------------------------------------------------------------|
 | `attribute.other` | Additional attributes to be used when performing directory searches |
@@ -80,6 +132,8 @@ ldap:
 |                   | Global filter is used if empty/blank/not set                        |

 ### Identity
+Base path: `ldap.identity`
+
 | Item     | Description                                                                                       |
 |----------|---------------------------------------------------------------------------------------------------|
 | `filter` | Specific user filter applied during identity search. Global filter is used if empty/blank/not set | 
--- a/docs/configure.md
+++ b/docs/configure.md
@@ -37,7 +37,7 @@ server:
 **WARNING:** mxisd might overwrite/adapt some values during launch. Those changes will not be reflected into copied keys.

 ## Categories
-For each category below, the base configuration path will be given, which needs to be appened to every configuration
+For each category below, the base configuration path will be given, which needs to be appended to every configuration
 item described.

 Example: if the base path was `basePath` and the following table was given:
--- a/docs/faq.md
+++ b/docs/faq.md
@@ -0,0 +1,65 @@
+# FAQ
+### Do I need to use mxisd if I run a Homeserver?
+No, but it is recommended, even if you don't use any backends or integration.
+
+In its default configuration, mxisd will talk to the central Matrix Identity servers and use other federated public
+servers when performing queries, giving you access to at least the same information as if you were not running it.
+
+It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
+privacy consequences, which is not the case with the central Matrix.org servers.
+
+So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
+simple features on top of it.
+
+### I already use the synapse LDAP3 auth provider, why should I care about mxisd?
+The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) is not longer maintained and
+only handles on specific flow: validate credentials at login.
+
+It does not:
+- Auto-provision user profiles
+- Integrate with Identity management
+- Integrate with Directory searches
+- Protect you against the username case sensitivites issues in synapse
+
+mxisd is a replacement and enhancement of it, offering coherent results in all areas, which LDAP3 auth provider
+does not.
+
+### Sydent is the official Identity server implementation of the Matrix team, why not use that?
+You can, but [sydent](https://github.com/matrix-org/sydent):
+- [should not be used and/or self-hosted](https://github.com/matrix-org/sydent/issues/22)
+- is not meant to be linked to a specific Homeserver / domain
+- cannot handle federation or proxy lookups, effectively isolating your users from the rest of the network
+- forces you to duplicate all your identity data, so people can be found by 3PIDs
+- forces users to enter all their emails and phone numbers manually in their profile
+
+So really, you should go with mxisd.
+
+### Will I loose access to the central Matrix.org/Vector.im Identity data if I use mxisd?
+In its default configuration, mxisd act as a proxy to Matrix.org/Vector.im. You will have access to the same data and
+behaviour than if you were using them directly. There is no downside in using mxisd with the default configuration.
+
+mxisd can also be configured not to talk to the central Identity servers if you wish.
+
+### I'm not sure I understand what an "Identity server" is supposed to be or do
+The current Identity service API is more a placeholder, as the Matrix devs did not have time so far to really work on
+what they want to do with that part of the ecosystem. Therefore, "Identity" is a misleading word currently.  
+Given the scope of the current Identity Service API, it would be best called "Invitation service".
+
+Because the current scope is so limited and no integration is done with the Homeserver, there was a big lack of features
+for groups/corporations/organisation. This is where mxisd comes in.
+
+mxisd implements the Identity Service API and also a set of features which are expected by regular users, truly living
+up to its "Identity server" name.
+
+### So mxisd is just a big hack! I don't want to use non-official features!
+mxisd primary concern is to always be compatible with the Matrix ecosystem and the Identity service API.  
+Whenever the API will be updated and/or enhanced, mxisd will follow, remaining 100% compatible with the ecosystem.
+
+We also directly talk with the Matrix developers to ensure all features we implement have their approval, and that we
+are in line with their vision of Identity management within the Matrix ecosystem.
+
+Therefore, using mxisd is a safe choice. It will be like using the central Matrix.org Identity servers, yet not closing
+the door to very nice enhancements and integrations.
+
+### Should I use mxisd if I don't host my own Homeserver?
+No
--- a/docs/features/authentication.md
+++ b/docs/features/authentication.md
@@ -1,6 +1,7 @@
 # Authentication
-Performed via [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)  
-Point the `endpoint` to mxisd internal IP on port 8090
+Authentication is an enchanced Identity feature of mxisd to ensure coherent and centralized identity management.
+
+It allows to use Identity stores configured in mxisd to authenticate users on your Homeserver.

 ## Overview
 ```
@@ -23,6 +24,31 @@ Point the `endpoint` to mxisd internal IP on port 8090
              |   user profiles          |    If valid credentials and supported by backend
              +--------------------------+
 ```
+Performed on [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)

-## Profile auto-fill
-To be documented
+## Getting started
+Authentication is possible by linking synapse and mxisd together using the REST auth module
+(also known as password provider).
+
+### Synapse
+- Install the [REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth).
+- Edit your synapse configuration:
+  - As described by the auth module documentation
+  - Set `endpoint` to `http://mxisdAddress:8090` - Replace `mxisdAddress` by an IP/host name that provides a direct
+  connection to mxisd.  
+  This **MUST NOT** be a public address, and SHOULD NOT go through a reverse proxy.
+- Restart synapse
+
+### mxisd
+- Configure and enable at least one [Identity store](../backends/)
+- Restart mxisd
+
+### Validate
+Login on the Homeserver using credentials present in your backend.
+
+## Next steps
+### Profile auto-fill
+Auto-filling user profile depends on two conditions:
+- The REST auth module is configured for it, which is the case by default
+- Your Identity store is configured to provide profile data. See your Identity store [documentation](../backends/) on
+how to enable the feature.
--- a/docs/features/directory-users.md
+++ b/docs/features/directory-users.md
@@ -9,6 +9,7 @@
    - [LDAP](#ldap)
    - [SQL](#sql)
    - [REST](#rest)
+- [Next steps](#next-steps)

 ## Description
 This feature allows you to search for existing and/or potential users that are already present in your Identity backend
@@ -165,3 +166,11 @@ For each query, `type` can be used to tell mxisd how to process the ID column:

 #### REST
 See the [dedicated document](../backends/rest.md)
+
+## Next steps
+### Homeserver results
+You can configure if the Homeserver should be queried at all when doing a directory search.  
+To disable Homeserver results, set the following in mxisd config file:
+```
+directory.exclude.homeserever: true
+```
--- a/docs/getting-started.md
+++ b/docs/getting-started.md
@@ -0,0 +1,113 @@
+# Getting started
+1. [Preparation](#preparation)
+2. [Install](#install)
+3. [Configure](#configure)
+4. [Integrate](#integrate)
+5. [Validate](#validate)
+6. [Next steps](#next-steps)
+
+Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
+talk to the central Matrix.org Identity service.  
+This will be a good ground work for further integration with your existing Identity stores.
+
+## Preparation
+You will need:
+- Homeserver
+- Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
+
+As synapse requires an HTTPS connection when talking to an Identity service, a reverse proxy is required as mxisd does
+not support HTTPS listener at this time.
+
+For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.  
+You can also use a dedicated domain for mxisd, but will not have access to some features.
+
+Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-io/mxisd/wiki/Gotchas#nating) if you use the same
+hostname.
+
+The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
+If you would like a high-level view of the infrastructure and how each feature is integrated, see the
+[dedicated document](architecture.md)
+
+## Install
+Install via:
+- [Debian package](install/debian.md)
+- [Docker image](install/docker.md)
+- [Sources](build.md)
+
+See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
+
+## Configure
+**NOTE**: please view the install instruction for your platform, as this step might be optional/handled for you.
+
+Create/edit a minimal configuration (see installer doc for the location):
+```
+matrix.domain: 'MyMatrixDomain.org'
+key.path: '/path/to/signing.key.file'
+storage.provider.sqlite.database: '/path/to/mxisd.db'
+```  
+- `matrix.domain` should be set to your Homeserver domain
+- `key.path` will store the signing keys, which must be kept safe!
+- `storage.provider.sqlite.database` is the location of the SQLite Database file which will hold state (invites, etc.)
+
+If your HS/mxisd hostname is not the same as your Matrix domain, configure `server.name`.  
+Complete configuration guide is available [here](configure.md).
+
+## Integrate
+For an overview of a typical mxisd infrastructure, see the [dedicated document](architecture.md)
+### Reverse proxy
+#### Apache2
+In the VirtualHost handling the domain with SSL, add the following line and replace `0.0.0.0` by the internal IP/hostname
+pointing to mxisd.  
+**This line MUST be present before the one for the homeserver!**
+```
+ProxyPass /_matrix/identity/ http://0.0.0.0:8090/_matrix/identity/
+```
+
+Typical VirtualHost configuration would be:
+```
+<VirtualHost *:443>
+    ServerName example.org
+    
+    ...
+    
+    ProxyPreserveHost on
+    ProxyPass /_matrix/identity/ http://10.1.2.3:8090/_matrix/identity/
+    ProxyPass /_matrix/ http://10.1.2.3:8008/_matrix/
+</VirtualHost>
+```
+
+### Synapse
+Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_servers` and restart synapse.  
+In a typical configuration, you would end up with something similair to:
+```
+trusted_third_party_id_servers:
+    - matrix.org
+    - vector.im
+    - example.org
+```
+It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is authoritative for your HS.
+
+## Validate
+Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
+the relevant hostname which you configured in your reverse proxy.  
+Invite `mxisd-lookup-test@kamax.io` to a room, which should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.  
+**NOTE:** you might not see a Matrix suggestion for the e-mail address, which is normal. Still proceed with the invite.
+  
+If it worked, it means you are up and running and can enjoy mxisd in its basic mode! Congratulations!  
+If it did not work, [get in touch](#support) and we'll do our best to get you started.
+
+You can now integrate mxisd further with your infrastructure using the various [features](README.md) guides.
+
+## Next steps
+Once your mxisd server is up and running, here are the next steps to further enhance and integrate your installation:
+
+Enable extra features:
+- [Federation](features/federation.md)
+- [Authenticate with synapse](features/authentication.md), profile auto-provisioning if you wish
+- [Directory search](features/directory-users.md)
+
+Use your Identity stores:
+- [LDAP / Samba / Active directory](backends/ldap.md)
+- [SQL Database](backends/sql.md)
+- [Website / Web service / Web app](backends/rest.md)
+- [Google Firebase](backends/firebase.md)
--- a/docs/install/docker.md
+++ b/docs/install/docker.md
@@ -5,10 +5,18 @@ Pull the latest stable image:
 docker pull kamax/mxisd
 ```

+## Configure
+On first run, simply using `MATRIX_DOMAIN` as an environment variable will create a default config for you.  
+You can also provide a configuration file named `mxisd.yaml` in the volume mapped to `/etc/mxisd` before starting your
+container.
+
 ## Run
-Run it (adapt volume paths to your host):
+Use the following command after adapting to your needs:
+- The `MATRIX_DOMAIN` environment variable to yours
+- The volumes host paths
+
 ```
-docker run --rm -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
+docker run --rm -e MATRIX_DOMAIN=example.org -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
 ```

 For more info, including the list of possible tags, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)
--- a/src/docker/start.sh
+++ b/src/docker/start.sh
@@ -1,2 +1,26 @@
 #!/bin/sh
+
+if ! [ -z "$CONF_FILE_PATH" ] && ! [ -f "CONF_FILE_PATH" ]; then
+    echo "Generating config file $CONF_FILE_PATH"
+    touch "CONF_FILE_PATH"
+
+    if ! [ -z "$MATRIX_DOMAIN" ]; then
+        echo "Setting matrix domain to $MATRIX_DOMAIN"
+        echo "matrix.domain: $MATRIX_DOMAIN" >> "$CONF_FILE_PATH"
+    fi
+
+    if ! [ -z "$SIGN_KEY_PATH" ]; then
+        echo "Setting signing key path to $SIGN_KEY_PATH"
+        echo "key.path: $SIGN_KEY_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    if ! [ -z "$SQLITE_DATABASE_PATH" ]; then
+        echo "Setting SQLite DB path to $SQLITE_DATABASE_PATH"
+        echo "storage.provider.sqlite.database: $SQLITE_DATABASE_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    echo "Starting mxisd..."
+    echo
+fi
+
 exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.config.location=/etc/mxisd/ -Dspring.config.name=mxisd -jar /mxisd.jar
--- a/src/main/java/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java
@@ -20,7 +20,11 @@

 package io.kamax.mxisd.backend.ldap;

+import com.google.i18n.phonenumbers.NumberParseException;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import io.kamax.matrix.ThreePidMedium;
 import io.kamax.matrix._MatrixID;
+import io.kamax.mxisd.ThreePid;
 import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
@@ -41,12 +45,18 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;

 import java.io.IOException;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;

@Component
 public class LdapAuthProvider extends LdapGenericBackend implements AuthenticatorProvider {

    private Logger log = LoggerFactory.getLogger(LdapAuthProvider.class);

+    private PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+
    @Autowired
    public LdapAuthProvider(LdapConfig cfg, MatrixConfig mxCfg) {
        super(cfg, mxCfg);
@@ -57,6 +67,21 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
        return getCfg().isEnabled();
    }

+    private Optional<String> getMsisdn(String phoneNumber) {
+        try { // FIXME export into dedicated ThreePid class within SDK (copy from Firebase Auth)
+            return Optional.of(phoneUtil.format(
+                    phoneUtil.parse(
+                            phoneNumber,
+                            null // No default region
+                    ),
+                    PhoneNumberUtil.PhoneNumberFormat.E164
+            ).substring(1)); // We want without the leading +
+        } catch (NumberParseException e) {
+            log.warn("Invalid phone number: {}", phoneNumber);
+            return Optional.empty();
+        }
+    }
+
    @Override
    public BackendAuthResult authenticate(_MatrixID mxid, String password) {
        log.info("Performing auth for {}", mxid);
@@ -74,7 +99,15 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato

            String userFilter = "(" + getUidAtt() + "=" + userFilterValue + ")";
            userFilter = buildWithFilter(userFilter, getCfg().getAuth().getFilter());
-            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, getUidAtt(), getAt().getName())) {
+
+            Set<String> attributes = new HashSet<>();
+            attributes.add(getUidAtt());
+            attributes.add(getAt().getName());
+            getAt().getThreepid().forEach((k, v) -> attributes.addAll(v));
+            String[] attArray = new String[attributes.size()];
+            attributes.toArray(attArray);
+
+            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, attArray)) {
                while (cursor.next()) {
                    Entry entry = cursor.get();
                    String dn = entry.getDn().getName();
@@ -99,7 +132,24 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
                    log.info("DN {} is a valid match", dn);

                    // TODO should we canonicalize the MXID?
-                    return BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                    BackendAuthResult result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                    log.info("Processing 3PIDs for profile");
+                    getAt().getThreepid().forEach((k, v) -> {
+                        log.info("Processing 3PID type {}", k);
+                        v.forEach(attId -> {
+                            List<String> values = getAttributes(entry, attId);
+                            log.info("\tAttribute {} has {} value(s)", attId, values.size());
+                            getAttributes(entry, attId).forEach(tpidValue -> {
+                                if (ThreePidMedium.PhoneNumber.is(k)) {
+                                    tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
+                                }
+                                result.withThreePid(new ThreePid(k, tpidValue));
+                            });
+                        });
+                    });
+
+                    log.info("Found {} 3PIDs", result.getProfile().getThreePids().size());
+                    return result;
                }
            } catch (CursorLdapReferralException e) {
                log.warn("Entity for {} is only available via referral, skipping", mxid);
--- a/src/main/java/io/kamax/mxisd/backend/ldap/LdapGenericBackend.java
+++ b/src/main/java/io/kamax/mxisd/backend/ldap/LdapGenericBackend.java
@@ -25,6 +25,7 @@ import io.kamax.mxisd.config.ldap.LdapAttributeConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.entry.Attribute;
+import org.apache.directory.api.ldap.model.entry.AttributeUtils;
 import org.apache.directory.api.ldap.model.entry.Entry;
 import org.apache.directory.api.ldap.model.exception.LdapException;
 import org.apache.directory.ldap.client.api.LdapConnection;
@@ -32,6 +33,9 @@ import org.apache.directory.ldap.client.api.LdapNetworkConnection;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;

+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Optional;
@@ -124,7 +128,6 @@ public abstract class LdapGenericBackend {
    public Optional<String> getAttribute(Entry entry, String attName) {
        Attribute attribute = entry.get(attName);
        if (attribute == null) {
-            log.info("DN {}: no attribute {}, skipping", entry.getDn(), attName);
            return Optional.empty();
        }

@@ -137,4 +140,22 @@ public abstract class LdapGenericBackend {
        return Optional.of(value);
    }

+    public List<String> getAttributes(Entry entry, String attName) {
+        List<String> values = new ArrayList<>();
+        javax.naming.directory.Attribute att = AttributeUtils.toAttributes(entry).get(attName);
+        if (att == null) {
+            return values;
+        }
+
+        try {
+            NamingEnumeration<?> list = att.getAll();
+            while (list.hasMore()) {
+                values.add(list.next().toString());
+            }
+        } catch (NamingException e) {
+            log.warn("Error while processing LDAP attribute {}, result could be incomplete!", attName, e);
+        }
+        return values;
+    }
+
 }
--- a/src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java
@@ -68,13 +68,16 @@ public class LdapThreePidProvider extends LdapGenericBackend implements IThreePi
    }

    private Optional<String> lookup(LdapConnection conn, String medium, String value) {
-        Optional<String> queryOpt = getCfg().getIdentity().getQuery(medium);
-        if (!queryOpt.isPresent()) {
+        Optional<String> tPidQueryOpt = getCfg().getIdentity().getQuery(medium);
+        if (!tPidQueryOpt.isPresent()) {
            log.warn("{} is not a configured 3PID type for LDAP lookup", medium);
            return Optional.empty();
        }

-        String searchQuery = queryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        // we merge 3PID specific query with global/specific filter, if one exists.
+        String tPidQuery = tPidQueryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        String searchQuery = buildWithFilter(tPidQuery, getCfg().getIdentity().getFilter());
+
        try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, getUidAtt())) {
            while (cursor.next()) {
                Entry entry = cursor.get();
--- a/src/main/java/io/kamax/mxisd/backend/sql/GenericSqlAuthProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/sql/GenericSqlAuthProvider.java
@@ -24,7 +24,7 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.ServerConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.invitation.InvitationManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -32,15 +32,15 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;

@Component
-public class SqlAuthProvider implements AuthenticatorProvider {
+public class GenericSqlAuthProvider implements AuthenticatorProvider {

-    private Logger log = LoggerFactory.getLogger(SqlAuthProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlAuthProvider.class);

    @Autowired
    private ServerConfig srvCfg;

    @Autowired
-    private SqlProviderConfig cfg;
+    private GenericSqlProviderConfig cfg;

    @Autowired
    private InvitationManager invMgr;
--- a/src/main/java/io/kamax/mxisd/backend/sql/GenericSqlDirectoryProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/sql/GenericSqlDirectoryProvider.java
@@ -22,8 +22,8 @@ package io.kamax.mxisd.backend.sql;

 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.SqlConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.IDirectoryProvider;
 import io.kamax.mxisd.exception.InternalServerError;
@@ -39,16 +39,16 @@ import java.util.Optional;

 import static io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult.Result;

-public abstract class SqlDirectoryProvider implements IDirectoryProvider {
+public abstract class GenericSqlDirectoryProvider implements IDirectoryProvider {

-    private Logger log = LoggerFactory.getLogger(SqlDirectoryProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlDirectoryProvider.class);

    protected SqlConfig cfg;
    private MatrixConfig mxCfg;

    private SqlConnectionPool pool;

-    public SqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
+    public GenericSqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
        this.cfg = cfg;
        this.pool = new SqlConnectionPool(cfg);
        this.mxCfg = mxCfg;
@@ -72,7 +72,7 @@ public abstract class SqlDirectoryProvider implements IDirectoryProvider {
        return Optional.of(item);
    }

-    public UserDirectorySearchResult search(String searchTerm, SqlProviderConfig.Query query) {
+    public UserDirectorySearchResult search(String searchTerm, GenericSqlProviderConfig.Query query) {
        try (Connection conn = pool.get()) {
            log.info("Will execute query: {}", query.getValue());
            try (PreparedStatement stmt = conn.prepareStatement(query.getValue())) {
--- a/src/main/java/io/kamax/mxisd/backend/sql/GenericSqlThreePidProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/sql/GenericSqlThreePidProvider.java
@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class GenericSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public GenericSqlThreePidProvider(GenericSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}
--- a/src/main/java/io/kamax/mxisd/backend/sql/SqlThreePidProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/sql/SqlThreePidProvider.java
@@ -22,7 +22,7 @@ package io.kamax.mxisd.backend.sql;

 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.SqlConfig;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
@@ -30,8 +30,6 @@ import io.kamax.mxisd.lookup.provider.IThreePidProvider;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;

 import java.sql.Connection;
 import java.sql.PreparedStatement;
@@ -41,18 +39,16 @@ import java.util.ArrayList;
 import java.util.List;
 import java.util.Optional;

-@Component
-public class SqlThreePidProvider implements IThreePidProvider {
+public abstract class SqlThreePidProvider implements IThreePidProvider {

    private Logger log = LoggerFactory.getLogger(SqlThreePidProvider.class);

-    private SqlProviderConfig cfg;
+    private SqlConfig cfg;
    private MatrixConfig mxCfg;

    private SqlConnectionPool pool;

-    @Autowired
-    public SqlThreePidProvider(SqlProviderConfig cfg, MatrixConfig mxCfg) {
+    public SqlThreePidProvider(SqlConfig cfg, MatrixConfig mxCfg) {
        this.cfg = cfg;
        this.pool = new SqlConnectionPool(cfg);
        this.mxCfg = mxCfg;
--- a/src/main/java/io/kamax/mxisd/backend/sql/SynapseSqlThreePidProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/sql/SynapseSqlThreePidProvider.java
@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class SynapseSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public SynapseSqlThreePidProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}
--- a/src/main/java/io/kamax/mxisd/backend/sql/SynapseSqliteDirectoryProvider.java
+++ b/src/main/java/io/kamax/mxisd/backend/sql/SynapseSqliteDirectoryProvider.java
@@ -21,7 +21,7 @@
 package io.kamax.mxisd.backend.sql;

 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
 import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang.StringUtils;
@@ -32,9 +32,7 @@ import java.sql.PreparedStatement;
 import java.sql.SQLException;

@Component
-public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
-
-    private SynapseSqlProviderConfig cfg;
+public class SynapseSqliteDirectoryProvider extends GenericSqlDirectoryProvider {

    @Autowired
    public SynapseSqliteDirectoryProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
@@ -42,7 +40,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {

        if (StringUtils.equals("sqlite", cfg.getType())) {
            String userId = "'@' || p.user_id || ':" + mxCfg.getDomain() + "'";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
            queries.getName().setValue(
                    "select " + userId + ", displayname from profiles p where displayname like ?");
            queries.getThreepid().setValue(
@@ -51,7 +49,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
                            "where t.address like ?");
        } else if (StringUtils.equals("postgresql", cfg.getType())) {
            String userId = "concat('@',p.user_id,':" + mxCfg.getDomain() + "')";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
            queries.getName().setValue(
                    "select " + userId + ", displayname from profiles p where displayname ilike ?");
            queries.getThreepid().setValue(
--- a/src/main/java/io/kamax/mxisd/config/DirectoryConfig.java
+++ b/src/main/java/io/kamax/mxisd/config/DirectoryConfig.java
@@ -0,0 +1,59 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+@Configuration
+@ConfigurationProperties("directory")
+public class DirectoryConfig {
+
+    private final transient Logger log = LoggerFactory.getLogger(DnsOverwriteConfig.class);
+
+    public static class Exclude {
+
+        private boolean homeserver;
+
+        public boolean getHomeserver() {
+            return homeserver;
+        }
+
+        public Exclude setHomeserver(boolean homeserver) {
+            this.homeserver = homeserver;
+            return this;
+        }
+
+    }
+
+    private Exclude exclude = new Exclude();
+
+    public Exclude getExclude() {
+        return exclude;
+    }
+
+    public void setExclude(Exclude exclude) {
+        this.exclude = exclude;
+    }
+
+}
--- a/src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java
+++ b/src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java
@@ -162,11 +162,14 @@ public class LdapConfig {
            throw new IllegalStateException("LDAP port is not valid");
        }

+        if (StringUtils.isBlank(conn.getBaseDn())) {
+            throw new ConfigurationException("ldap.connection.baseDn");
+        }
+
        if (StringUtils.isBlank(attribute.getUid().getType())) {
            throw new IllegalStateException("Attribute UID Type cannot be empty");
        }

-
        if (StringUtils.isBlank(attribute.getUid().getValue())) {
            throw new IllegalStateException("Attribute UID value cannot be empty");
        }
--- a/src/main/java/io/kamax/mxisd/config/sql/GenericSqlProviderConfig.java
+++ b/src/main/java/io/kamax/mxisd/config/sql/GenericSqlProviderConfig.java
@@ -24,21 +24,14 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 import org.springframework.context.annotation.Primary;

-import javax.annotation.PostConstruct;
-
@Configuration
@ConfigurationProperties("sql")
@Primary
-public class SqlProviderConfig extends SqlConfig {
+public class GenericSqlProviderConfig extends SqlConfig {

    @Override
    protected String getProviderName() {
        return "Generic SQL";
    }

-    @PostConstruct
-    public void build() {
-        super.build();
-    }
-
-}
+}
--- a/src/main/java/io/kamax/mxisd/config/sql/SqlConfig.java
+++ b/src/main/java/io/kamax/mxisd/config/sql/SqlConfig.java
@@ -4,6 +4,7 @@ import io.kamax.mxisd.util.GsonUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;

+import javax.annotation.PostConstruct;
 import java.util.HashMap;
 import java.util.Map;

@@ -36,22 +37,22 @@ public abstract class SqlConfig {

    public static class Type {

-        private SqlProviderConfig.Query name = new SqlProviderConfig.Query();
-        private SqlProviderConfig.Query threepid = new SqlProviderConfig.Query();
+        private GenericSqlProviderConfig.Query name = new GenericSqlProviderConfig.Query();
+        private GenericSqlProviderConfig.Query threepid = new GenericSqlProviderConfig.Query();

-        public SqlProviderConfig.Query getName() {
+        public GenericSqlProviderConfig.Query getName() {
            return name;
        }

-        public void setName(SqlProviderConfig.Query name) {
+        public void setName(GenericSqlProviderConfig.Query name) {
            this.name = name;
        }

-        public SqlProviderConfig.Query getThreepid() {
+        public GenericSqlProviderConfig.Query getThreepid() {
            return threepid;
        }

-        public void setThreepid(SqlProviderConfig.Query threepid) {
+        public void setThreepid(GenericSqlProviderConfig.Query threepid) {
            this.threepid = threepid;
        }

@@ -74,7 +75,7 @@ public abstract class SqlConfig {
    public static class Directory {

        private Boolean enabled;
-        private SqlProviderConfig.Type query = new SqlProviderConfig.Type();
+        private GenericSqlProviderConfig.Type query = new GenericSqlProviderConfig.Type();

        public Boolean isEnabled() {
            return enabled;
@@ -84,11 +85,11 @@ public abstract class SqlConfig {
            this.enabled = enabled;
        }

-        public SqlProviderConfig.Type getQuery() {
+        public GenericSqlProviderConfig.Type getQuery() {
            return query;
        }

-        public void setQuery(SqlProviderConfig.Type query) {
+        public void setQuery(GenericSqlProviderConfig.Type query) {
            this.query = query;
        }

@@ -138,9 +139,9 @@ public abstract class SqlConfig {
    private boolean enabled;
    private String type;
    private String connection;
-    private SqlProviderConfig.Auth auth = new SqlProviderConfig.Auth();
-    private SqlProviderConfig.Directory directory = new SqlProviderConfig.Directory();
-    private SqlProviderConfig.Identity identity = new SqlProviderConfig.Identity();
+    private GenericSqlProviderConfig.Auth auth = new GenericSqlProviderConfig.Auth();
+    private GenericSqlProviderConfig.Directory directory = new GenericSqlProviderConfig.Directory();
+    private GenericSqlProviderConfig.Identity identity = new GenericSqlProviderConfig.Identity();

    public boolean isEnabled() {
        return enabled;
@@ -166,35 +167,33 @@ public abstract class SqlConfig {
        this.connection = connection;
    }

-    public SqlProviderConfig.Auth getAuth() {
+    public GenericSqlProviderConfig.Auth getAuth() {
        return auth;
    }

-    public void setAuth(SqlProviderConfig.Auth auth) {
+    public void setAuth(GenericSqlProviderConfig.Auth auth) {
        this.auth = auth;
    }

-    public SqlProviderConfig.Directory getDirectory() {
+    public GenericSqlProviderConfig.Directory getDirectory() {
        return directory;
    }

-    public void setDirectory(SqlProviderConfig.Directory directory) {
+    public void setDirectory(GenericSqlProviderConfig.Directory directory) {
        this.directory = directory;
    }

-    public SqlProviderConfig.Identity getIdentity() {
+    public GenericSqlProviderConfig.Identity getIdentity() {
        return identity;
    }

-    public void setIdentity(SqlProviderConfig.Identity identity) {
+    public void setIdentity(GenericSqlProviderConfig.Identity identity) {
        this.identity = identity;
    }

    protected abstract String getProviderName();

-    public void build() {
-        log.info("--- " + getProviderName() + " Provider config ---");
-
+    protected void doBuild() {
        if (getAuth().isEnabled() == null) {
            getAuth().setEnabled(isEnabled());
        }
@@ -206,6 +205,13 @@ public abstract class SqlConfig {
        if (getIdentity().isEnabled() == null) {
            getIdentity().setEnabled(isEnabled());
        }
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- " + getProviderName() + " Provider config ---");
+
+        doBuild();

        log.info("Enabled: {}", isEnabled());
        if (isEnabled()) {
@@ -214,6 +220,7 @@ public abstract class SqlConfig {
            log.info("Auth enabled: {}", getAuth().isEnabled());
            log.info("Directory queries: {}", GsonUtil.build().toJson(getDirectory().getQuery()));
            log.info("Identity type: {}", getIdentity().getType());
+            log.info("3PID mapping query: {}", getIdentity().getQuery());
            log.info("Identity medium queries: {}", GsonUtil.build().toJson(getIdentity().getMedium()));
        }
    }
--- a/src/main/java/io/kamax/mxisd/config/sql/synapse/SynapseSqlProviderConfig.java
+++ b/src/main/java/io/kamax/mxisd/config/sql/synapse/SynapseSqlProviderConfig.java
@@ -21,6 +21,7 @@
 package io.kamax.mxisd.config.sql.synapse;

 import io.kamax.mxisd.config.sql.SqlConfig;
+import org.apache.commons.lang.StringUtils;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;

@@ -36,8 +37,23 @@ public class SynapseSqlProviderConfig extends SqlConfig {
    }

    @PostConstruct
-    public void build() {
-        super.build();
+    public void doBuild() {
+        super.doBuild();
+        // FIXME check that the DB is not the mxisd one
+        // See https://matrix.to/#/!NPRUEisLjcaMtHIzDr:kamax.io/$1509377583327omXkC:kamax.io
+
+        getAuth().setEnabled(false); // Synapse does the auth, we only act as a directory/identity service.
+
+        if (getDirectory().isEnabled()) {
+            //FIXME set default queries for name and threepid
+        }
+
+        if (getIdentity().isEnabled()) {
+            if (StringUtils.isBlank(getIdentity().getType())) {
+                getIdentity().setType("mxid");
+                getIdentity().setQuery("SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?");
+            }
+        }
    }

 }
--- a/src/main/java/io/kamax/mxisd/controller/DefaultExceptionHandler.java
+++ b/src/main/java/io/kamax/mxisd/controller/DefaultExceptionHandler.java
@@ -55,20 +55,29 @@ public class DefaultExceptionHandler {
    }

    @ExceptionHandler(InternalServerError.class)
-    public String handle(HttpServletRequest req, InternalServerError e, HttpServletResponse response) {
+    public String handle(HttpServletRequest request, HttpServletResponse response, InternalServerError e) {
        if (StringUtils.isNotBlank(e.getInternalReason())) {
            log.error("Reference #{} - {}", e.getReference(), e.getInternalReason());
        } else {
            log.error("Reference #{}", e);
        }

-        return handleGeneric(req, e, response);
+        return handleGeneric(request, response, e);
+    }
+
+    @ExceptionHandler(FeatureNotAvailable.class)
+    public String handle(HttpServletRequest request, HttpServletResponse response, FeatureNotAvailable e) {
+        if (StringUtils.isNotBlank(e.getInternalReason())) {
+            log.error("Feature not available: {}", e.getInternalReason());
+        }
+
+        return handleGeneric(request, response, e);
    }

    @ExceptionHandler(MatrixException.class)
-    public String handleGeneric(HttpServletRequest req, MatrixException e, HttpServletResponse response) {
+    public String handleGeneric(HttpServletRequest request, HttpServletResponse response, MatrixException e) {
        response.setStatus(e.getStatus());
-        return handle(req, e.getErrorCode(), e.getError());
+        return handle(request, e.getErrorCode(), e.getError());
    }

    @ResponseStatus(HttpStatus.BAD_REQUEST)
--- a/src/main/java/io/kamax/mxisd/controller/identity/v1/SessionController.java
+++ b/src/main/java/io/kamax/mxisd/controller/identity/v1/SessionController.java
@@ -23,7 +23,6 @@ package io.kamax.mxisd.controller.identity.v1;
 import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.config.ViewConfig;
 import io.kamax.mxisd.controller.identity.v1.remote.RemoteIdentityAPIv1;
-import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.session.SessionMananger;
 import io.kamax.mxisd.session.ValidationResult;
 import org.slf4j.Logger;
@@ -36,7 +35,8 @@ import org.springframework.web.bind.annotation.RequestParam;

 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
-import java.io.IOException;
+import java.net.MalformedURLException;
+import java.net.URL;

 import static org.springframework.web.bind.annotation.RequestMethod.GET;

@@ -69,15 +69,15 @@ class SessionController {
        ValidationResult r = mgr.validate(sid, secret, token);
        log.info("Session {} was validated", sid);
        if (r.getNextUrl().isPresent()) {
-            String url = srvCfg.getPublicUrl() + r.getNextUrl().get();
-            log.info("Session {} validation: next URL is present, redirecting to {}", sid, url);
+            String url = r.getNextUrl().get();
            try {
-                response.sendRedirect(url);
-                return "";
-            } catch (IOException e) {
-                log.warn("Unable to redirect user to {}", url);
-                throw new InternalServerError(e);
+                url = new URL(url).toString();
+            } catch (MalformedURLException e) {
+                log.info("Session next URL {} is not a valid one, will prepend public URL {}", url, srvCfg.getPublicUrl());
+                url = srvCfg.getPublicUrl() + r.getNextUrl().get();
            }
+            log.info("Session {} validation: next URL is present, redirecting to {}", sid, url);
+            return "redirect:" + url;
        } else {
            if (r.isCanRemote()) {
                String url = srvCfg.getPublicUrl() + RemoteIdentityAPIv1.getRequestToken(r.getSession().getId(), r.getSession().getSecret());
--- a/src/main/java/io/kamax/mxisd/directory/DirectoryManager.java
+++ b/src/main/java/io/kamax/mxisd/directory/DirectoryManager.java
@@ -23,6 +23,7 @@ package io.kamax.mxisd.directory;
 import com.google.gson.Gson;
 import com.google.gson.JsonSyntaxException;
 import io.kamax.matrix.MatrixErrorInfo;
+import io.kamax.mxisd.config.DirectoryConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchRequest;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.dns.ClientDnsOverwrite;
@@ -31,6 +32,7 @@ import io.kamax.mxisd.exception.MatrixException;
 import io.kamax.mxisd.util.GsonUtil;
 import io.kamax.mxisd.util.RestClientUtils;
 import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpPost;
 import org.apache.http.client.utils.URIBuilder;
@@ -53,6 +55,7 @@ public class DirectoryManager {

    private Logger log = LoggerFactory.getLogger(DirectoryManager.class);

+    private DirectoryConfig cfg;
    private List<IDirectoryProvider> providers;

    private ClientDnsOverwrite dns;
@@ -60,7 +63,8 @@ public class DirectoryManager {
    private Gson gson;

    @Autowired
-    public DirectoryManager(List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+    public DirectoryManager(DirectoryConfig cfg, List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+        this.cfg = cfg;
        this.dns = dns;
        this.client = HttpClients.custom().setUserAgent("mxisd").build(); //FIXME centralize
        this.gson = GsonUtil.build();
@@ -75,32 +79,41 @@ public class DirectoryManager {
        log.info("Original request URL: {}", target);
        UserDirectorySearchResult result = new UserDirectorySearchResult();

-        URIBuilder builder = dns.transform(target);
-        log.info("Querying HS at {}", builder);
-        builder.setParameter("access_token", accessToken);
-        HttpPost req = RestClientUtils.post(
-                builder.toString(),
-                new UserDirectorySearchRequest(query));
-        try (CloseableHttpResponse res = client.execute(req)) {
-            int status = res.getStatusLine().getStatusCode();
-            Charset charset = ContentType.getOrDefault(res.getEntity()).getCharset();
-            String body = IOUtils.toString(res.getEntity().getContent(), charset);
+        if (cfg.getExclude().getHomeserver()) {
+            log.info("Skipping HS directory data, disabled in config");
+        } else {
+            URIBuilder builder = dns.transform(target);
+            log.info("Querying HS at {}", builder);
+            builder.setParameter("access_token", accessToken);
+            HttpPost req = RestClientUtils.post(
+                    builder.toString(),
+                    new UserDirectorySearchRequest(query));
+            try (CloseableHttpResponse res = client.execute(req)) {
+                int status = res.getStatusLine().getStatusCode();
+                Charset charset = ContentType.getOrDefault(res.getEntity()).getCharset();
+                String body = IOUtils.toString(res.getEntity().getContent(), charset);

-            if (status != 200) {
-                MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
-                throw new MatrixException(status, info.getErrcode(), info.getError());
-            }
+                if (status != 200) {
+                    MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
+                    if (StringUtils.equals("M_UNRECOGNIZED", info.getErrcode())) { // FIXME no hardcoding, use Enum
+                        log.warn("Homeserver does not support Directory feature, skipping");
+                    } else {
+                        log.error("Homeserver returned an error while performing directory search");
+                        throw new MatrixException(status, info.getErrcode(), info.getError());
+                    }
+                }

-            UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
-            log.info("Found {} match(es) in HS for '{}'", resultHs.getResults().size(), query);
-            result.getResults().addAll(resultHs.getResults());
-            if (resultHs.isLimited()) {
-                result.setLimited(true);
+                UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
+                log.info("Found {} match(es) in HS for '{}'", resultHs.getResults().size(), query);
+                result.getResults().addAll(resultHs.getResults());
+                if (resultHs.isLimited()) {
+                    result.setLimited(true);
+                }
+            } catch (JsonSyntaxException e) {
+                throw new InternalServerError("Invalid JSON reply from the HS: " + e.getMessage());
+            } catch (IOException e) {
+                throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
            }
-        } catch (JsonSyntaxException e) {
-            throw new InternalServerError("Invalid JSON reply from the HS: " + e.getMessage());
-        } catch (IOException e) {
-            throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
        }

        for (IDirectoryProvider provider : providers) {
--- a/src/main/java/io/kamax/mxisd/exception/FeatureNotAvailable.java
+++ b/src/main/java/io/kamax/mxisd/exception/FeatureNotAvailable.java
@@ -0,0 +1,43 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.exception;
+
+import org.apache.http.HttpStatus;
+
+public class FeatureNotAvailable extends MatrixException {
+
+    private String internalReason;
+
+    public FeatureNotAvailable(String internalReason) {
+        super(
+                HttpStatus.SC_INTERNAL_SERVER_ERROR,
+                "M_NOT_AVAILABLE",
+                "This action is currently not available. Contact your administrator to enable it."
+        );
+
+        this.internalReason = internalReason;
+    }
+
+    public String getInternalReason() {
+        return internalReason;
+    }
+
+}
--- a/src/main/java/io/kamax/mxisd/lookup/strategy/RecursivePriorityLookupStrategy.java
+++ b/src/main/java/io/kamax/mxisd/lookup/strategy/RecursivePriorityLookupStrategy.java
@@ -53,7 +53,10 @@ public class RecursivePriorityLookupStrategy implements LookupStrategy {
    public RecursivePriorityLookupStrategy(RecursiveLookupConfig cfg, List<IThreePidProvider> providers, IBridgeFetcher bridge) {
        this.cfg = cfg;
        this.bridge = bridge;
-        this.providers = providers.stream().filter(IThreePidProvider::isEnabled).collect(Collectors.toList());
+        this.providers = providers.stream().filter(p -> {
+            log.info("3PID Provider {} is enabled: {}", p.getClass().getSimpleName(), p.isEnabled());
+            return p.isEnabled();
+        }).collect(Collectors.toList());
    }

    @PostConstruct
--- a/src/main/java/io/kamax/mxisd/matrix/IdentityServerUtils.java
+++ b/src/main/java/io/kamax/mxisd/matrix/IdentityServerUtils.java
@@ -4,6 +4,7 @@ import com.google.gson.JsonElement;
 import com.google.gson.JsonParseException;
 import com.google.gson.JsonParser;
 import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.xbill.DNS.*;
@@ -28,6 +29,10 @@ public class IdentityServerUtils {
    private static JsonParser parser = new JsonParser();

    public static boolean isUsable(String remote) {
+        if (StringUtils.isBlank(remote)) {
+            return false;
+        }
+
        try {
            // FIXME use Apache HTTP client
            HttpURLConnection rootSrvConn = (HttpURLConnection) new URL(
@@ -54,7 +59,7 @@ public class IdentityServerUtils {
            }

            return true;
-        } catch (IOException | JsonParseException e) {
+        } catch (IllegalArgumentException | IOException | JsonParseException e) {
            log.info("{} is not a usable Identity Server: {}", remote, e.getMessage());
            return false;
        }
--- a/src/main/java/io/kamax/mxisd/notification/NotificationManager.java
+++ b/src/main/java/io/kamax/mxisd/notification/NotificationManager.java
@@ -46,16 +46,14 @@ public class NotificationManager {
        this.handlers = new HashMap<>();
        handlers.forEach(h -> {
            log.info("Found handler {} for medium {}", h.getId(), h.getMedium());
-            String handlerId = cfg.getHandler().get(h.getMedium());
-            if (StringUtils.isBlank(handlerId) || StringUtils.equals(handlerId, h.getId())) {
+            String handlerId = cfg.getHandler().getOrDefault(h.getMedium(), "raw");
+            if (StringUtils.equals(handlerId, h.getId())) {
                this.handlers.put(h.getMedium(), h);
            }
        });

        log.info("--- Notification handler ---");
-        this.handlers.forEach((k, v) -> {
-            log.info("\tHandler for {}: {}", k, v.getId());
-        });
+        this.handlers.forEach((k, v) -> log.info("\tHandler for {}: {}", k, v.getId()));
    }

    private INotificationHandler ensureMedium(String medium) {
--- a/src/main/java/io/kamax/mxisd/session/SessionMananger.java
+++ b/src/main/java/io/kamax/mxisd/session/SessionMananger.java
@@ -272,9 +272,13 @@ public class SessionMananger {

        List<String> servers = mxCfg.getIdentity().getServers(policy.getToRemote().getServer());
        if (servers.isEmpty()) {
-            throw new InternalServerError();
+            throw new FeatureNotAvailable("Remote 3PID sessions are enabled but server list is " +
+                    "misconstrued (invalid ID or empty list");
        }
-        String url = IdentityServerUtils.findIsUrlForDomain(servers.get(0)).orElseThrow(InternalServerError::new);
+
+        String is = servers.get(0);
+        String url = IdentityServerUtils.findIsUrlForDomain(is)
+                .orElseThrow(() -> new InternalServerError(is + " could not be resolved to an Identity server"));
        log.info("Will use IS endpoint {}", url);

        String remoteSecret = session.isRemote() ? session.getRemoteSecret() : RandomStringUtils.randomAlphanumeric(16);
@@ -283,13 +287,17 @@ public class SessionMananger {
        body.addProperty("client_secret", remoteSecret);
        body.addProperty(session.getThreePid().getMedium(), session.getThreePid().getAddress());
        body.addProperty("send_attempt", session.increaseAndGetRemoteAttempt());
-        try {
-            Phonenumber.PhoneNumber msisdn = phoneUtil.parse("+" + session.getThreePid().getAddress(), null);
-            String country = phoneUtil.getRegionCodeForNumber(msisdn).toUpperCase();
-            body.addProperty("phone_number", phoneUtil.format(msisdn, PhoneNumberUtil.PhoneNumberFormat.NATIONAL));
-            body.addProperty("country", country);
-        } catch (NumberParseException e) {
-            throw new InternalServerError(e);
+        if (ThreePidMedium.PhoneNumber.is(session.getThreePid().getMedium())) {
+            try {
+                Phonenumber.PhoneNumber msisdn = phoneUtil.parse("+" + session.getThreePid().getAddress(), null);
+                String country = phoneUtil.getRegionCodeForNumber(msisdn).toUpperCase();
+                body.addProperty("phone_number", phoneUtil.format(msisdn, PhoneNumberUtil.PhoneNumberFormat.NATIONAL));
+                body.addProperty("country", country);
+            } catch (NumberParseException e) {
+                throw new InternalServerError(e);
+            }
+        } else {
+            body.addProperty(session.getThreePid().getMedium(), session.getThreePid().getAddress());
        }

        log.info("Requesting remote session with attempt {}", session.getRemoteAttempt());
--- a/src/main/java/io/kamax/mxisd/storage/ormlite/OrmLiteSqliteStorage.java
+++ b/src/main/java/io/kamax/mxisd/storage/ormlite/OrmLiteSqliteStorage.java
@@ -146,8 +146,8 @@ public class OrmLiteSqliteStorage implements IStorage {
        return withCatcher(() -> {
            List<ThreePidSessionDao> daoList = sessionDao.queryForMatchingArgs(new ThreePidSessionDao(tpid, secret));
            if (daoList.size() > 1) {
-                log.error("Lookup for 3PID Session {}:{} returned more than one result");
-                throw new InternalServerError();
+                throw new InternalServerError("Lookup for 3PID Session " +
+                        tpid + " returned more than one result");
            }

            if (daoList.isEmpty()) {
--- a/src/main/java/io/kamax/mxisd/threepid/connector/email/EmailSendGridNotificationHandler.java
+++ b/src/main/java/io/kamax/mxisd/threepid/connector/email/EmailSendGridNotificationHandler.java
@@ -26,11 +26,13 @@ import io.kamax.matrix.ThreePidMedium;
 import io.kamax.mxisd.config.MatrixConfig;
 import io.kamax.mxisd.config.ServerConfig;
 import io.kamax.mxisd.config.threepid.connector.EmailSendGridConfig;
+import io.kamax.mxisd.exception.FeatureNotAvailable;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.notification.INotificationHandler;
 import io.kamax.mxisd.threepid.notification.PlaceholderNotificationGenerator;
 import io.kamax.mxisd.threepid.session.IThreePidSession;
 import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
@@ -118,6 +120,11 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
    }

    private void send(String recipient, Email email) {
+        if (StringUtils.isBlank(cfg.getIdentity().getFrom())) {
+            throw new FeatureNotAvailable("3PID Email identity: sender address is empty - " +
+                    "You must set a value for notifications to work");
+        }
+
        try {
            email.addTo(recipient);
            email.setFrom(cfg.getIdentity().getFrom());
--- a/src/main/java/io/kamax/mxisd/threepid/connector/email/EmailSmtpConnector.java
+++ b/src/main/java/io/kamax/mxisd/threepid/connector/email/EmailSmtpConnector.java
@@ -23,6 +23,7 @@ package io.kamax.mxisd.threepid.connector.email;
 import com.sun.mail.smtp.SMTPTransport;
 import io.kamax.matrix.ThreePidMedium;
 import io.kamax.mxisd.config.threepid.connector.EmailSmtpConfig;
+import io.kamax.mxisd.exception.FeatureNotAvailable;
 import io.kamax.mxisd.exception.InternalServerError;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang.StringUtils;
@@ -66,6 +67,11 @@ public class EmailSmtpConnector implements IEmailConnector {

    @Override
    public void send(String senderAddress, String senderName, String recipient, String content) {
+        if (StringUtils.isBlank(senderAddress)) {
+            throw new FeatureNotAvailable("3PID Email identity: sender address is empty - " +
+                    "You must set a value for notifications to work");
+        }
+
        if (StringUtils.isBlank(content)) {
            throw new InternalServerError("Notification content is empty");
        }
--- a/src/main/resources/application.yaml
+++ b/src/main/resources/application.yaml
@@ -1,3 +1,8 @@
+# DO NOT USE THIS FILE AS-IS FOR YOUR INITIAL CONFIGURATION
+# ONLY TAKE THE SPECIFIC SECTION YOU WANT TO CONFIGURE
+#
+# For more information about configuration, visit https://github.com/kamax-io/mxisd/blob/master/docs/configure.md
+
 spring:
  main:
    banner-mode: 'off'
@@ -222,10 +227,14 @@ view:
 storage:
  backend: 'sqlite'

+directory:
+  exclude:
+    homeserver: false
+
 ---
 spring:
  profiles: systemd

 logging:
  pattern:
-    console: '%d{.SSS}${LOG_LEVEL_PATTERN:%5p} [%15.15t] %35.35logger{34} : %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}'
+    console: '%d{.SSS} ${LOG_LEVEL_PATTERN:%5p} [%15.15t] %35.35logger{34} : %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}'
Author	SHA1	Message	Date
Maxime Dor	7fff2448a1	Improve the docker experience - Only one env variable to configure on first usage - Auto-generate a default config - Improve doc	2017-12-16 19:58:11 +01:00
Maxime Dor	6571ff76b1	Take LDAP filter into account when doing 3PID lookups	2017-12-16 19:17:22 +01:00
Maxime Dor	16690a0329	Enforce baseDn for LDAP provider	2017-12-06 20:31:06 +01:00
Maxime Dor	6ac593f0fa	Fix typo	2017-12-02 20:13:25 +01:00
Maxime Dor	1581ab9e07	Better logic (cosmetic) for default 3PID notification providers	2017-11-30 02:44:21 +01:00
Maxime Dor	a1adca72e8	Properly select raw notification handler by default	2017-11-26 16:30:42 +01:00
Maxime Dor	e2b3920840	Directory: document HS exclusion config option	2017-11-22 19:40:16 +01:00
Maxime Dor	aaa742f6d2	LDAP: Properly handle multi-value attributes	2017-11-17 16:51:16 +01:00
Maxime Dor	959feb686c	Improve auth doc	2017-11-16 22:35:16 +01:00
Maxime Dor	d9c5c5056a	Improve getting started wording	2017-11-15 21:01:33 +01:00
Maxime Dor	83fafdcfeb	Add config option to disable HS lookup for directory searches	2017-11-06 11:16:22 +01:00
Maxime Dor	e916ecd08b	Properly handle Synapse as an Identity provider	2017-10-30 17:43:22 +01:00
Maxime Dor	1461d8ef6c	Add fixme to prevent using mxisd DB as synapse identity store	2017-10-30 16:42:00 +01:00
Maxime Dor	19c1214e4a	Fix release version extraction from git tags	2017-10-25 00:40:43 +02:00
Maxime Dor	b976f69c39	Fix systemd log format	2017-10-17 15:59:16 +02:00
Maxime Dor	3675da4a0f	Specify that LDAP now supports profile auto-fill	2017-10-11 21:08:10 +02:00
Max Dor	077955d538	Set theme jekyll-theme-cayman	2017-10-09 19:18:37 +02:00
Maxime Dor	9af0cd3615	Support for profile auto-fill with LDAP	2017-10-08 04:22:38 +02:00
Maxime Dor	2bf68538c3	Fix typo, broken links	2017-10-08 03:41:45 +02:00
Maxime Dor	6c02e478d9	Add donate link	2017-10-07 18:50:17 +02:00
Maxime Dor	284da779f9	Fix typo	2017-10-07 18:39:13 +02:00
Maxime Dor	af161296b3	Be clear about profile auto-fill in LDAP auth	2017-10-07 18:27:47 +02:00
Maxime Dor	6317acd7fc	Add missing links	2017-10-07 18:02:47 +02:00
Maxime Dor	30260af1f2	More documentation	2017-10-07 17:59:55 +02:00
Maxime Dor	3b697e86ac	Various error handling improvements and user feedback	2017-10-07 06:15:57 +02:00
Maxime Dor	b4f0645257	Handle Homeservers not implementing the directory endpoint	2017-10-06 14:12:43 +02:00
Maxime Dor	0e48edf86e	Properly handle session next url	2017-10-06 14:10:08 +02:00
Maxime Dor	7e92bfa474	Add warning not to use built-in configuration file	2017-10-05 21:42:02 +02:00
Maxime Dor	851e0c9d94	Properly build remote 3PID token request	2017-10-05 16:04:29 +02:00
Maxime Dor	ac1cbc4265	Fix redirect with thymleaf	2017-10-05 16:03:59 +02:00
Maxime Dor	62711ee12e	Improve README	2017-10-02 20:07:22 +02:00