Improve the docker experience

- Only one env variable to configure on first usage
- Auto-generate a default config
- Improve doc

Dockerfile

@@ -6,6 +6,11 @@ EXPOSE 8090
 
 ADD build/libs/mxisd.jar /mxisd.jar
 ADD src/docker/start.sh /start.sh
+RUN mkdir -p /var/mxisd
 
 ENV JAVA_OPTS=""
+ENV CONF_FILE_PATH="/etc/mxisd/mxisd.yaml"
+ENV SIGN_KEY_PATH="/var/mxisd/sign.key"
+ENV SQLITE_DATABASE_PATH="/var/mxisd/mxisd.db"
+
 CMD [ "/start.sh" ]

README.md

@@ -18,7 +18,7 @@ It is specifically designed to connect to an Identity store (AD/Samba/LDAP, SQL
 and ease the integration of the Matrix ecosystem with an existing infrastructure, or to build a new one using lasting
 tools.
 
-The core principle of mxisd is to map between Matrix IDs and 3PIDs (Thrid-party Identifiers) for the Homeserver and its
+The core principle of mxisd is to map between Matrix IDs and 3PIDs (Third-party Identifiers) for the Homeserver and its
 users. 3PIDs can be anything that identify a user, like:
 - Full name
 - Email address

build.gradle

@@ -47,7 +47,7 @@ String gitVersion() {
     def versionPattern = Pattern.compile("v(\\d+\\.)?(\\d+\\.)?(\\d+)(-.*)?")
     ByteArrayOutputStream out = new ByteArrayOutputStream()
     exec {
-        commandLine = ['git', 'describe', '--always', '--dirty']
+        commandLine = ['git', 'describe', '--tags', '--always', '--dirty']
         standardOutput = out
     }
     def v = out.toString().replace(System.lineSeparator(), '')

docs/_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-cayman

docs/backends/ldap.md

@@ -43,7 +43,8 @@ ldap.attribute.name: 'cn'
 ```
 
 You can also change the attribute lists for 3PID, like email or phone numbers.  
-The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67) for emails and phone number:
+The following example would overwrite the [default list of attributes](../../src/main/resources/application.yaml#L67)
+for emails and phone number:
 ```
 ldap.attribute.threepid.email:
   - 'mail'
@@ -65,7 +66,8 @@ of the Configuration below.
 No further configuration is needed to enable authentication with LDAP once globally enabled and configured.  
 You have the possiblity to use a different query filter if you wish, see Configuration below.
 
-Profile auto-fill is not yet supported but is a top priority.
+Profile auto-fill is enabled by default. It will use the `name` and `threepid` configuration options to get a lit of
+attributes to be used to build the user profile to pass on to synapse during authentication.
 
 ## Directory
 No further configuration is needed to enable directory with LDAP once globally enabled and configured.

docs/features/authentication.md

@@ -27,22 +27,27 @@ It allows to use Identity stores configured in mxisd to authenticate users on yo
 Performed on [synapse with REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth/blob/master/README.md)
 
 ## Getting started
+Authentication is possible by linking synapse and mxisd together using the REST auth module
+(also known as password provider).
+
 ### Synapse
-You will need:
+- Install the [REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth).
+- Edit your synapse configuration:
+  - As described by the auth module documentation
+  - Set `endpoint` to `http://mxisdAddress:8090` - Replace `mxisdAddress` by an IP/host name that provides a direct
+  connection to mxisd.  
+  This **MUST NOT** be a public address, and SHOULD NOT go through a reverse proxy.
+- Restart synapse
+
+### mxisd
 - Configure and enable at least one [Identity store](../backends/)
-- Install the [REST auth module](https://github.com/kamax-io/matrix-synapse-rest-auth)
+- Restart mxisd
 
-Once installed, edit your synapse configuration as described for the auth module:
+### Validate
-- Set `endpoint` to `http://mxisdAddress:8090` - Replace `mxisdAddress` to an internal IP/Hostname.
+Login on the Homeserver using credentials present in your backend.
-- If you want to avoid [known issues](https://github.com/matrix-org/matrix-doc/issues/586) with lower/upper case
-usernames, set `enforceLowercase` in the REST config to `true`.
 
-**IMPORTANT**: if this is a new installation, it is highly recommended to enforce lowercase, as it is not possible to
+## Next steps
-workaround the bug at a later date and will cause issues with invites, searches, authentication.
+### Profile auto-fill
-
-Restart synapse and login on the Homeserver using credentials present in your backend.  
-
-## Profile auto-fill
 Auto-filling user profile depends on two conditions:
 - The REST auth module is configured for it, which is the case by default
 - Your Identity store is configured to provide profile data. See your Identity store [documentation](../backends/) on

docs/features/directory-users.md

@@ -9,6 +9,7 @@
     - [LDAP](#ldap)
     - [SQL](#sql)
     - [REST](#rest)
+- [Next steps](#next-steps)
 
 ## Description
 This feature allows you to search for existing and/or potential users that are already present in your Identity backend
@@ -165,3 +166,11 @@ For each query, `type` can be used to tell mxisd how to process the ID column:
 
 #### REST
 See the [dedicated document](../backends/rest.md)
+
+## Next steps
+### Homeserver results
+You can configure if the Homeserver should be queried at all when doing a directory search.  
+To disable Homeserver results, set the following in mxisd config file:
+```
+directory.exclude.homeserever: true
+```

docs/getting-started.md

@@ -37,6 +37,8 @@ Install via:
 See the [Latest release](https://github.com/kamax-io/mxisd/releases/latest) for links to each.
 
 ## Configure
+**NOTE**: please view the install instruction for your platform, as this step might be optional/handled for you.
+
 Create/edit a minimal configuration (see installer doc for the location):
 ```
 matrix.domain: 'MyMatrixDomain.org'
@@ -54,7 +56,8 @@ Complete configuration guide is available [here](configure.md).
 For an overview of a typical mxisd infrastructure, see the [dedicated document](architecture.md)
 ### Reverse proxy
 #### Apache2
-In the VirtualHost handling the domain with SSL, add the following line and replace `0.0.0.0` by the right address/host.  
+In the VirtualHost handling the domain with SSL, add the following line and replace `0.0.0.0` by the internal IP/hostname
+pointing to mxisd.  
 **This line MUST be present before the one for the homeserver!**
 ```
 ProxyPass /_matrix/identity/ http://0.0.0.0:8090/_matrix/identity/
@@ -82,7 +85,7 @@ trusted_third_party_id_servers:
     - vector.im
     - example.org
 ```
-It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is allowed by synapse. 
+It is recommended to remove `matrix.org` and `vector.im` so only your own Identity server is authoritative for your HS.
 
 ## Validate
 Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by

docs/install/docker.md

@@ -5,10 +5,18 @@ Pull the latest stable image:
 docker pull kamax/mxisd
 ```
 
+## Configure
+On first run, simply using `MATRIX_DOMAIN` as an environment variable will create a default config for you.  
+You can also provide a configuration file named `mxisd.yaml` in the volume mapped to `/etc/mxisd` before starting your
+container.
+
 ## Run
-Run it (adapt volume paths to your host):
+Use the following command after adapting to your needs:
+- The `MATRIX_DOMAIN` environment variable to yours
+- The volumes host paths
+
 ```
-docker run --rm -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
+docker run --rm -e MATRIX_DOMAIN=example.org -v /data/mxisd/etc:/etc/mxisd -v /data/mxisd/var:/var/mxisd -p 8090:8090 -t kamax/mxisd
 ```
 
 For more info, including the list of possible tags, see [the public repository](https://hub.docker.com/r/kamax/mxisd/)

src/docker/start.sh

@@ -1,2 +1,26 @@
 #!/bin/sh
+
+if ! [ -z "$CONF_FILE_PATH" ] && ! [ -f "CONF_FILE_PATH" ]; then
+    echo "Generating config file $CONF_FILE_PATH"
+    touch "CONF_FILE_PATH"
+
+    if ! [ -z "$MATRIX_DOMAIN" ]; then
+        echo "Setting matrix domain to $MATRIX_DOMAIN"
+        echo "matrix.domain: $MATRIX_DOMAIN" >> "$CONF_FILE_PATH"
+    fi
+
+    if ! [ -z "$SIGN_KEY_PATH" ]; then
+        echo "Setting signing key path to $SIGN_KEY_PATH"
+        echo "key.path: $SIGN_KEY_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    if ! [ -z "$SQLITE_DATABASE_PATH" ]; then
+        echo "Setting SQLite DB path to $SQLITE_DATABASE_PATH"
+        echo "storage.provider.sqlite.database: $SQLITE_DATABASE_PATH" >> "$CONF_FILE_PATH"
+    fi
+
+    echo "Starting mxisd..."
+    echo
+fi
+
 exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -Dspring.config.location=/etc/mxisd/ -Dspring.config.name=mxisd -jar /mxisd.jar

src/main/java/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java

@@ -46,6 +46,7 @@ import org.springframework.stereotype.Component;
 
 import java.io.IOException;
 import java.util.HashSet;
+import java.util.List;
 import java.util.Optional;
 import java.util.Set;
 
@@ -133,14 +134,20 @@ public class LdapAuthProvider extends LdapGenericBackend implements Authenticato
                     // TODO should we canonicalize the MXID?
                     BackendAuthResult result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
                     log.info("Processing 3PIDs for profile");
-                    getAt().getThreepid().forEach((k, v) -> v.forEach(attId -> {
+                    getAt().getThreepid().forEach((k, v) -> {
-                        getAttribute(entry, attId).ifPresent(tpidValue -> {
+                        log.info("Processing 3PID type {}", k);
-                            if (ThreePidMedium.PhoneNumber.is(k)) {
+                        v.forEach(attId -> {
-                                tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
+                            List<String> values = getAttributes(entry, attId);
-                            }
+                            log.info("\tAttribute {} has {} value(s)", attId, values.size());
-                            result.withThreePid(new ThreePid(k, tpidValue));
+                            getAttributes(entry, attId).forEach(tpidValue -> {
+                                if (ThreePidMedium.PhoneNumber.is(k)) {
+                                    tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
+                                }
+                                result.withThreePid(new ThreePid(k, tpidValue));
+                            });
                         });
-                    }));
+                    });
+
                     log.info("Found {} 3PIDs", result.getProfile().getThreePids().size());
                     return result;
                 }

src/main/java/io/kamax/mxisd/backend/ldap/LdapGenericBackend.java

@@ -25,6 +25,7 @@ import io.kamax.mxisd.config.ldap.LdapAttributeConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.entry.Attribute;
+import org.apache.directory.api.ldap.model.entry.AttributeUtils;
 import org.apache.directory.api.ldap.model.entry.Entry;
 import org.apache.directory.api.ldap.model.exception.LdapException;
 import org.apache.directory.ldap.client.api.LdapConnection;
@@ -32,6 +33,9 @@ import org.apache.directory.ldap.client.api.LdapNetworkConnection;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.naming.NamingEnumeration;
+import javax.naming.NamingException;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.List;
 import java.util.Optional;
@@ -124,7 +128,6 @@ public abstract class LdapGenericBackend {
     public Optional<String> getAttribute(Entry entry, String attName) {
         Attribute attribute = entry.get(attName);
         if (attribute == null) {
-            log.info("DN {}: no attribute {}, skipping", entry.getDn(), attName);
             return Optional.empty();
         }
 
@@ -137,4 +140,22 @@ public abstract class LdapGenericBackend {
         return Optional.of(value);
     }
 
+    public List<String> getAttributes(Entry entry, String attName) {
+        List<String> values = new ArrayList<>();
+        javax.naming.directory.Attribute att = AttributeUtils.toAttributes(entry).get(attName);
+        if (att == null) {
+            return values;
+        }
+
+        try {
+            NamingEnumeration<?> list = att.getAll();
+            while (list.hasMore()) {
+                values.add(list.next().toString());
+            }
+        } catch (NamingException e) {
+            log.warn("Error while processing LDAP attribute {}, result could be incomplete!", attName, e);
+        }
+        return values;
+    }
+
 }

src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java

@@ -68,13 +68,16 @@ public class LdapThreePidProvider extends LdapGenericBackend implements IThreePi
     }
 
     private Optional<String> lookup(LdapConnection conn, String medium, String value) {
-        Optional<String> queryOpt = getCfg().getIdentity().getQuery(medium);
+        Optional<String> tPidQueryOpt = getCfg().getIdentity().getQuery(medium);
-        if (!queryOpt.isPresent()) {
+        if (!tPidQueryOpt.isPresent()) {
             log.warn("{} is not a configured 3PID type for LDAP lookup", medium);
             return Optional.empty();
         }
 
-        String searchQuery = queryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        // we merge 3PID specific query with global/specific filter, if one exists.
+        String tPidQuery = tPidQueryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
+        String searchQuery = buildWithFilter(tPidQuery, getCfg().getIdentity().getFilter());
+
         try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, getUidAtt())) {
             while (cursor.next()) {
                 Entry entry = cursor.get();

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlAuthProvider.java

@@ -24,7 +24,7 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.ServerConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.invitation.InvitationManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -32,15 +32,15 @@ import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;
 
 @Component
-public class SqlAuthProvider implements AuthenticatorProvider {
+public class GenericSqlAuthProvider implements AuthenticatorProvider {
 
-    private Logger log = LoggerFactory.getLogger(SqlAuthProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlAuthProvider.class);
 
     @Autowired
     private ServerConfig srvCfg;
 
     @Autowired
-    private SqlProviderConfig cfg;
+    private GenericSqlProviderConfig cfg;
 
     @Autowired
     private InvitationManager invMgr;

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlDirectoryProvider.java

@@ -22,8 +22,8 @@ package io.kamax.mxisd.backend.sql;
 
 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.SqlConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.directory.IDirectoryProvider;
 import io.kamax.mxisd.exception.InternalServerError;
@@ -39,16 +39,16 @@ import java.util.Optional;
 
 import static io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult.Result;
 
-public abstract class SqlDirectoryProvider implements IDirectoryProvider {
+public abstract class GenericSqlDirectoryProvider implements IDirectoryProvider {
 
-    private Logger log = LoggerFactory.getLogger(SqlDirectoryProvider.class);
+    private Logger log = LoggerFactory.getLogger(GenericSqlDirectoryProvider.class);
 
     protected SqlConfig cfg;
     private MatrixConfig mxCfg;
 
     private SqlConnectionPool pool;
 
-    public SqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
+    public GenericSqlDirectoryProvider(SqlConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.pool = new SqlConnectionPool(cfg);
         this.mxCfg = mxCfg;
@@ -72,7 +72,7 @@ public abstract class SqlDirectoryProvider implements IDirectoryProvider {
         return Optional.of(item);
     }
 
-    public UserDirectorySearchResult search(String searchTerm, SqlProviderConfig.Query query) {
+    public UserDirectorySearchResult search(String searchTerm, GenericSqlProviderConfig.Query query) {
         try (Connection conn = pool.get()) {
             log.info("Will execute query: {}", query.getValue());
             try (PreparedStatement stmt = conn.prepareStatement(query.getValue())) {

src/main/java/io/kamax/mxisd/backend/sql/GenericSqlThreePidProvider.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class GenericSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public GenericSqlThreePidProvider(GenericSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/sql/SqlThreePidProvider.java

@@ -22,7 +22,7 @@ package io.kamax.mxisd.backend.sql;
 
 import io.kamax.matrix.MatrixID;
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.SqlConfig;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidMapping;
@@ -30,8 +30,6 @@ import io.kamax.mxisd.lookup.provider.IThreePidProvider;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
-import org.springframework.beans.factory.annotation.Autowired;
-import org.springframework.stereotype.Component;
 
 import java.sql.Connection;
 import java.sql.PreparedStatement;
@@ -41,18 +39,16 @@ import java.util.ArrayList;
 import java.util.List;
 import java.util.Optional;
 
-@Component
+public abstract class SqlThreePidProvider implements IThreePidProvider {
-public class SqlThreePidProvider implements IThreePidProvider {
 
     private Logger log = LoggerFactory.getLogger(SqlThreePidProvider.class);
 
-    private SqlProviderConfig cfg;
+    private SqlConfig cfg;
     private MatrixConfig mxCfg;
 
     private SqlConnectionPool pool;
 
-    @Autowired
+    public SqlThreePidProvider(SqlConfig cfg, MatrixConfig mxCfg) {
-    public SqlThreePidProvider(SqlProviderConfig cfg, MatrixConfig mxCfg) {
         this.cfg = cfg;
         this.pool = new SqlConnectionPool(cfg);
         this.mxCfg = mxCfg;

src/main/java/io/kamax/mxisd/backend/sql/SynapseSqlThreePidProvider.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.sql;
+
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.stereotype.Component;
+
+@Component
+public class SynapseSqlThreePidProvider extends SqlThreePidProvider {
+
+    @Autowired
+    public SynapseSqlThreePidProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
+        super(cfg, mxCfg);
+    }
+
+}

src/main/java/io/kamax/mxisd/backend/sql/SynapseSqliteDirectoryProvider.java

@@ -21,7 +21,7 @@
 package io.kamax.mxisd.backend.sql;
 
 import io.kamax.mxisd.config.MatrixConfig;
-import io.kamax.mxisd.config.sql.SqlProviderConfig;
+import io.kamax.mxisd.config.sql.GenericSqlProviderConfig;
 import io.kamax.mxisd.config.sql.synapse.SynapseSqlProviderConfig;
 import io.kamax.mxisd.exception.ConfigurationException;
 import org.apache.commons.lang.StringUtils;
@@ -32,9 +32,7 @@ import java.sql.PreparedStatement;
 import java.sql.SQLException;
 
 @Component
-public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
+public class SynapseSqliteDirectoryProvider extends GenericSqlDirectoryProvider {
-
-    private SynapseSqlProviderConfig cfg;
 
     @Autowired
     public SynapseSqliteDirectoryProvider(SynapseSqlProviderConfig cfg, MatrixConfig mxCfg) {
@@ -42,7 +40,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
 
         if (StringUtils.equals("sqlite", cfg.getType())) {
             String userId = "'@' || p.user_id || ':" + mxCfg.getDomain() + "'";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
             queries.getName().setValue(
                     "select " + userId + ", displayname from profiles p where displayname like ?");
             queries.getThreepid().setValue(
@@ -51,7 +49,7 @@ public class SynapseSqliteDirectoryProvider extends SqlDirectoryProvider {
                             "where t.address like ?");
         } else if (StringUtils.equals("postgresql", cfg.getType())) {
             String userId = "concat('@',p.user_id,':" + mxCfg.getDomain() + "')";
-            SqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
+            GenericSqlProviderConfig.Type queries = cfg.getDirectory().getQuery();
             queries.getName().setValue(
                     "select " + userId + ", displayname from profiles p where displayname ilike ?");
             queries.getThreepid().setValue(

src/main/java/io/kamax/mxisd/config/DirectoryConfig.java

@@ -0,0 +1,59 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Maxime Dor
+ *
+ * https://max.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+@Configuration
+@ConfigurationProperties("directory")
+public class DirectoryConfig {
+
+    private final transient Logger log = LoggerFactory.getLogger(DnsOverwriteConfig.class);
+
+    public static class Exclude {
+
+        private boolean homeserver;
+
+        public boolean getHomeserver() {
+            return homeserver;
+        }
+
+        public Exclude setHomeserver(boolean homeserver) {
+            this.homeserver = homeserver;
+            return this;
+        }
+
+    }
+
+    private Exclude exclude = new Exclude();
+
+    public Exclude getExclude() {
+        return exclude;
+    }
+
+    public void setExclude(Exclude exclude) {
+        this.exclude = exclude;
+    }
+
+}

src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java

@@ -162,11 +162,14 @@ public class LdapConfig {
             throw new IllegalStateException("LDAP port is not valid");
         }
 
+        if (StringUtils.isBlank(conn.getBaseDn())) {
+            throw new ConfigurationException("ldap.connection.baseDn");
+        }
+
         if (StringUtils.isBlank(attribute.getUid().getType())) {
             throw new IllegalStateException("Attribute UID Type cannot be empty");
         }
 
-
         if (StringUtils.isBlank(attribute.getUid().getValue())) {
             throw new IllegalStateException("Attribute UID value cannot be empty");
         }

src/main/java/io/kamax/mxisd/config/sql/GenericSqlProviderConfig.java

@@ -24,21 +24,14 @@ import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 import org.springframework.context.annotation.Primary;
 
-import javax.annotation.PostConstruct;
-
 @Configuration
 @ConfigurationProperties("sql")
 @Primary
-public class SqlProviderConfig extends SqlConfig {
+public class GenericSqlProviderConfig extends SqlConfig {
 
     @Override
     protected String getProviderName() {
         return "Generic SQL";
     }
 
-    @PostConstruct
-    public void build() {
-        super.build();
-    }
-
 }

src/main/java/io/kamax/mxisd/config/sql/SqlConfig.java

@@ -4,6 +4,7 @@ import io.kamax.mxisd.util.GsonUtil;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import javax.annotation.PostConstruct;
 import java.util.HashMap;
 import java.util.Map;
 
@@ -36,22 +37,22 @@ public abstract class SqlConfig {
 
     public static class Type {
 
-        private SqlProviderConfig.Query name = new SqlProviderConfig.Query();
+        private GenericSqlProviderConfig.Query name = new GenericSqlProviderConfig.Query();
-        private SqlProviderConfig.Query threepid = new SqlProviderConfig.Query();
+        private GenericSqlProviderConfig.Query threepid = new GenericSqlProviderConfig.Query();
 
-        public SqlProviderConfig.Query getName() {
+        public GenericSqlProviderConfig.Query getName() {
             return name;
         }
 
-        public void setName(SqlProviderConfig.Query name) {
+        public void setName(GenericSqlProviderConfig.Query name) {
             this.name = name;
         }
 
-        public SqlProviderConfig.Query getThreepid() {
+        public GenericSqlProviderConfig.Query getThreepid() {
             return threepid;
         }
 
-        public void setThreepid(SqlProviderConfig.Query threepid) {
+        public void setThreepid(GenericSqlProviderConfig.Query threepid) {
             this.threepid = threepid;
         }
 
@@ -74,7 +75,7 @@ public abstract class SqlConfig {
     public static class Directory {
 
         private Boolean enabled;
-        private SqlProviderConfig.Type query = new SqlProviderConfig.Type();
+        private GenericSqlProviderConfig.Type query = new GenericSqlProviderConfig.Type();
 
         public Boolean isEnabled() {
             return enabled;
@@ -84,11 +85,11 @@ public abstract class SqlConfig {
             this.enabled = enabled;
         }
 
-        public SqlProviderConfig.Type getQuery() {
+        public GenericSqlProviderConfig.Type getQuery() {
             return query;
         }
 
-        public void setQuery(SqlProviderConfig.Type query) {
+        public void setQuery(GenericSqlProviderConfig.Type query) {
             this.query = query;
         }
 
@@ -138,9 +139,9 @@ public abstract class SqlConfig {
     private boolean enabled;
     private String type;
     private String connection;
-    private SqlProviderConfig.Auth auth = new SqlProviderConfig.Auth();
+    private GenericSqlProviderConfig.Auth auth = new GenericSqlProviderConfig.Auth();
-    private SqlProviderConfig.Directory directory = new SqlProviderConfig.Directory();
+    private GenericSqlProviderConfig.Directory directory = new GenericSqlProviderConfig.Directory();
-    private SqlProviderConfig.Identity identity = new SqlProviderConfig.Identity();
+    private GenericSqlProviderConfig.Identity identity = new GenericSqlProviderConfig.Identity();
 
     public boolean isEnabled() {
         return enabled;
@@ -166,35 +167,33 @@ public abstract class SqlConfig {
         this.connection = connection;
     }
 
-    public SqlProviderConfig.Auth getAuth() {
+    public GenericSqlProviderConfig.Auth getAuth() {
         return auth;
     }
 
-    public void setAuth(SqlProviderConfig.Auth auth) {
+    public void setAuth(GenericSqlProviderConfig.Auth auth) {
         this.auth = auth;
     }
 
-    public SqlProviderConfig.Directory getDirectory() {
+    public GenericSqlProviderConfig.Directory getDirectory() {
         return directory;
     }
 
-    public void setDirectory(SqlProviderConfig.Directory directory) {
+    public void setDirectory(GenericSqlProviderConfig.Directory directory) {
         this.directory = directory;
     }
 
-    public SqlProviderConfig.Identity getIdentity() {
+    public GenericSqlProviderConfig.Identity getIdentity() {
         return identity;
     }
 
-    public void setIdentity(SqlProviderConfig.Identity identity) {
+    public void setIdentity(GenericSqlProviderConfig.Identity identity) {
         this.identity = identity;
     }
 
     protected abstract String getProviderName();
 
-    public void build() {
+    protected void doBuild() {
-        log.info("--- " + getProviderName() + " Provider config ---");
-
         if (getAuth().isEnabled() == null) {
             getAuth().setEnabled(isEnabled());
         }
@@ -206,6 +205,13 @@ public abstract class SqlConfig {
         if (getIdentity().isEnabled() == null) {
             getIdentity().setEnabled(isEnabled());
         }
+    }
+
+    @PostConstruct
+    public void build() {
+        log.info("--- " + getProviderName() + " Provider config ---");
+
+        doBuild();
 
         log.info("Enabled: {}", isEnabled());
         if (isEnabled()) {
@@ -214,6 +220,7 @@ public abstract class SqlConfig {
             log.info("Auth enabled: {}", getAuth().isEnabled());
             log.info("Directory queries: {}", GsonUtil.build().toJson(getDirectory().getQuery()));
             log.info("Identity type: {}", getIdentity().getType());
+            log.info("3PID mapping query: {}", getIdentity().getQuery());
             log.info("Identity medium queries: {}", GsonUtil.build().toJson(getIdentity().getMedium()));
         }
     }

src/main/java/io/kamax/mxisd/config/sql/synapse/SynapseSqlProviderConfig.java

@@ -21,6 +21,7 @@
 package io.kamax.mxisd.config.sql.synapse;
 
 import io.kamax.mxisd.config.sql.SqlConfig;
+import org.apache.commons.lang.StringUtils;
 import org.springframework.boot.context.properties.ConfigurationProperties;
 import org.springframework.context.annotation.Configuration;
 
@@ -36,8 +37,23 @@ public class SynapseSqlProviderConfig extends SqlConfig {
     }
 
     @PostConstruct
-    public void build() {
+    public void doBuild() {
-        super.build();
+        super.doBuild();
+        // FIXME check that the DB is not the mxisd one
+        // See https://matrix.to/#/!NPRUEisLjcaMtHIzDr:kamax.io/$1509377583327omXkC:kamax.io
+
+        getAuth().setEnabled(false); // Synapse does the auth, we only act as a directory/identity service.
+
+        if (getDirectory().isEnabled()) {
+            //FIXME set default queries for name and threepid
+        }
+
+        if (getIdentity().isEnabled()) {
+            if (StringUtils.isBlank(getIdentity().getType())) {
+                getIdentity().setType("mxid");
+                getIdentity().setQuery("SELECT user_id AS uid FROM user_threepids WHERE medium = ? AND address = ?");
+            }
+        }
     }
 
 }

src/main/java/io/kamax/mxisd/directory/DirectoryManager.java

@@ -23,6 +23,7 @@ package io.kamax.mxisd.directory;
 import com.google.gson.Gson;
 import com.google.gson.JsonSyntaxException;
 import io.kamax.matrix.MatrixErrorInfo;
+import io.kamax.mxisd.config.DirectoryConfig;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchRequest;
 import io.kamax.mxisd.controller.directory.v1.io.UserDirectorySearchResult;
 import io.kamax.mxisd.dns.ClientDnsOverwrite;
@@ -54,6 +55,7 @@ public class DirectoryManager {
 
     private Logger log = LoggerFactory.getLogger(DirectoryManager.class);
 
+    private DirectoryConfig cfg;
     private List<IDirectoryProvider> providers;
 
     private ClientDnsOverwrite dns;
@@ -61,7 +63,8 @@ public class DirectoryManager {
     private Gson gson;
 
     @Autowired
-    public DirectoryManager(List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+    public DirectoryManager(DirectoryConfig cfg, List<IDirectoryProvider> providers, ClientDnsOverwrite dns) {
+        this.cfg = cfg;
         this.dns = dns;
         this.client = HttpClients.custom().setUserAgent("mxisd").build(); //FIXME centralize
         this.gson = GsonUtil.build();
@@ -76,37 +79,41 @@ public class DirectoryManager {
         log.info("Original request URL: {}", target);
         UserDirectorySearchResult result = new UserDirectorySearchResult();
 
-        URIBuilder builder = dns.transform(target);
+        if (cfg.getExclude().getHomeserver()) {
-        log.info("Querying HS at {}", builder);
+            log.info("Skipping HS directory data, disabled in config");
-        builder.setParameter("access_token", accessToken);
+        } else {
-        HttpPost req = RestClientUtils.post(
+            URIBuilder builder = dns.transform(target);
-                builder.toString(),
+            log.info("Querying HS at {}", builder);
-                new UserDirectorySearchRequest(query));
+            builder.setParameter("access_token", accessToken);
-        try (CloseableHttpResponse res = client.execute(req)) {
+            HttpPost req = RestClientUtils.post(
-            int status = res.getStatusLine().getStatusCode();
+                    builder.toString(),
-            Charset charset = ContentType.getOrDefault(res.getEntity()).getCharset();
+                    new UserDirectorySearchRequest(query));
-            String body = IOUtils.toString(res.getEntity().getContent(), charset);
+            try (CloseableHttpResponse res = client.execute(req)) {
+                int status = res.getStatusLine().getStatusCode();
+                Charset charset = ContentType.getOrDefault(res.getEntity()).getCharset();
+                String body = IOUtils.toString(res.getEntity().getContent(), charset);
 
-            if (status != 200) {
+                if (status != 200) {
-                MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
+                    MatrixErrorInfo info = gson.fromJson(body, MatrixErrorInfo.class);
-                if (StringUtils.equals("M_UNRECOGNIZED", info.getErrcode())) { // FIXME no hardcoding, use Enum
+                    if (StringUtils.equals("M_UNRECOGNIZED", info.getErrcode())) { // FIXME no hardcoding, use Enum
-                    log.warn("Homeserver does not support Directory feature, skipping");
+                        log.warn("Homeserver does not support Directory feature, skipping");
-                } else {
+                    } else {
-                    log.error("Homeserver returned an error while performing directory search");
+                        log.error("Homeserver returned an error while performing directory search");
-                    throw new MatrixException(status, info.getErrcode(), info.getError());
+                        throw new MatrixException(status, info.getErrcode(), info.getError());
+                    }
                 }
-            }
 
-            UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
+                UserDirectorySearchResult resultHs = gson.fromJson(body, UserDirectorySearchResult.class);
-            log.info("Found {} match(es) in HS for '{}'", resultHs.getResults().size(), query);
+                log.info("Found {} match(es) in HS for '{}'", resultHs.getResults().size(), query);
-            result.getResults().addAll(resultHs.getResults());
+                result.getResults().addAll(resultHs.getResults());
-            if (resultHs.isLimited()) {
+                if (resultHs.isLimited()) {
-                result.setLimited(true);
+                    result.setLimited(true);
+                }
+            } catch (JsonSyntaxException e) {
+                throw new InternalServerError("Invalid JSON reply from the HS: " + e.getMessage());
+            } catch (IOException e) {
+                throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
             }
-        } catch (JsonSyntaxException e) {
-            throw new InternalServerError("Invalid JSON reply from the HS: " + e.getMessage());
-        } catch (IOException e) {
-            throw new InternalServerError("Unable to query the HS: I/O error: " + e.getMessage());
         }
 
         for (IDirectoryProvider provider : providers) {

src/main/java/io/kamax/mxisd/lookup/strategy/RecursivePriorityLookupStrategy.java

@@ -53,7 +53,10 @@ public class RecursivePriorityLookupStrategy implements LookupStrategy {
     public RecursivePriorityLookupStrategy(RecursiveLookupConfig cfg, List<IThreePidProvider> providers, IBridgeFetcher bridge) {
         this.cfg = cfg;
         this.bridge = bridge;
-        this.providers = providers.stream().filter(IThreePidProvider::isEnabled).collect(Collectors.toList());
+        this.providers = providers.stream().filter(p -> {
+            log.info("3PID Provider {} is enabled: {}", p.getClass().getSimpleName(), p.isEnabled());
+            return p.isEnabled();
+        }).collect(Collectors.toList());
     }
 
     @PostConstruct

src/main/java/io/kamax/mxisd/notification/NotificationManager.java

@@ -46,16 +46,14 @@ public class NotificationManager {
         this.handlers = new HashMap<>();
         handlers.forEach(h -> {
             log.info("Found handler {} for medium {}", h.getId(), h.getMedium());
-            String handlerId = cfg.getHandler().get(h.getMedium());
+            String handlerId = cfg.getHandler().getOrDefault(h.getMedium(), "raw");
-            if (StringUtils.isBlank(handlerId) || StringUtils.equals(handlerId, h.getId())) {
+            if (StringUtils.equals(handlerId, h.getId())) {
                 this.handlers.put(h.getMedium(), h);
             }
         });
 
         log.info("--- Notification handler ---");
-        this.handlers.forEach((k, v) -> {
+        this.handlers.forEach((k, v) -> log.info("\tHandler for {}: {}", k, v.getId()));
-            log.info("\tHandler for {}: {}", k, v.getId());
-        });
     }
 
     private INotificationHandler ensureMedium(String medium) {

src/main/resources/application.yaml

@@ -227,10 +227,14 @@ view:
 storage:
   backend: 'sqlite'
 
+directory:
+  exclude:
+    homeserver: false
+
 ---
 spring:
   profiles: systemd
 
 logging:
   pattern:
-    console: '%d{.SSS}${LOG_LEVEL_PATTERN:%5p} [%15.15t] %35.35logger{34} : %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}'
+    console: '%d{.SSS} ${LOG_LEVEL_PATTERN:%5p} [%15.15t] %35.35logger{34} : %m%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}'