Add missing header

README.md

@@ -14,13 +14,14 @@ mxisd - Federated Matrix Identity Server
 
 # Overview
 mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with [enhanced features](#features).
-As an enhanced Identity service, it implements the [Matrix Identity service API](https://kamax.io/matrix/api/identity_service/unstable.html)
+As an enhanced Identity service, it implements the [Identity service API](https://matrix.org/docs/spec/identity_service/r0.1.0.html)
 and several [extra features](#features) that greatly enhance user experience within Matrix.
 It is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built in a
 single coherent product.
   
 mxisd is specifically designed to connect to an existing on-premise Identity store (AD/Samba/LDAP, SQL Database,
 Web services/app, etc.) and ease the integration of a Matrix infrastructure within an existing one.  
+Check [our FAQ entry](docs/faq.md#what-kind-of-setup-is-mxisd-really-designed-for) to know if mxisd is a good fit for you.
 
 The core principle of mxisd is to map between Matrix IDs and 3PIDs (Third-Party IDentifiers) for the Homeserver and its
 users. 3PIDs can be anything that uniquely and globally identify a user, like:
@@ -33,15 +34,15 @@ users. 3PIDs can be anything that uniquely and globally identify a user, like:
 If you are unfamiliar with the Identity vocabulary and concepts in Matrix, **please read this [introduction](docs/concepts.md)**.
 
 # Features
-[Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://kamax.io/matrix/api/identity_service/unstable.html#general-principles):
+[Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://matrix.org/docs/spec/identity_service/r0.1.0.html#general-principles):
 - Search for people by 3PID using its own Identity stores
-  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#association-lookup))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#association-lookup))
 - Invite people to rooms by 3PID using its own Identity stores, with notifications to the invitee (Email, SMS, etc.)
-  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#post-matrix-identity-api-v1-store-invite))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-store-invite))
 - Allow users to add 3PIDs to their settings/profile
-  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#establishing-associations))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#establishing-associations))
 - Register accounts on your Homeserver with 3PIDs
-  ([Spec](https://kamax.io/matrix/api/identity_service/unstable.html#establishing-associations))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#establishing-associations))
 
 As an enhanced Identity service:
 - [Federation](docs/features/federation.md): Use a recursive lookup mechanism when searching and inviting people by 3PID,
@@ -67,6 +68,8 @@ As an enhanced Identity service:
 - Users can directly find each other using whatever attribute is relevant within your Identity store
 - Federate your Identity server so you can discover others and/or others can discover you
 
+Also, check [our FAQ entry](docs/faq.md#what-kind-of-setup-is-mxisd-really-designed-for) to know if mxisd is a good fit for you.
+
 # Getting started
 See the [dedicated document](docs/getting-started.md)
 

docs/faq.md

@@ -16,6 +16,18 @@ of the Matrix protocol is required for some advanced features.
 If all fails, come over to [the project room](https://matrix.to/#/#mxisd:kamax.io) and we'll do our best to get you
 started and answer questions you might have.
 
+### What kind of setup is mxisd really designed for?
+mxisd is primarily designed for setups that:
+- [Care for their privacy](https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy)
+- Have their own [domains](https://en.wikipedia.org/wiki/Domain_name)
+- Use those domains for their email addresses and all other services
+- Already have an [Identity store](stores/README.md), typically [LDAP-based](stores/ldap.md).
+
+If you meet all the conditions, then you are the prime use case we designed mxisd for. 
+
+If you meet some of the conditions, but not all, mxisd will still be a good fit for you but you won't fully enjoy all its
+features.
+
 ### Do I need to use mxisd if I run a Homeserver?
 No, but it is strongly recommended, even if you don't use any Identity store or integration.
 
@@ -23,9 +35,6 @@ In its default configuration, mxisd uses other federated public servers when per
 It can also [be configured](features/identity.md#lookups) to use the central matrix.org servers, giving you access to at
 least the same information as if you were not running it.
 
-It will also give your users a choice to make their 3PIDs available publicly, ensuring they are made aware of the
-privacy consequences, which is not the case with the central Matrix.org servers.
-
 So mxisd is like your gatekeeper and guardian angel. It does not change what you already know, just adds some nice
 simple features on top of it.
 
@@ -47,13 +56,14 @@ Accounts cannot currently migrate/move from one server to another.
 See a [brief explanation document](concepts.md) about Matrix and mxisd concepts and vocabulary.
 
 ### I already use the synapse LDAP3 auth provider. Why should I care about mxisd?
-The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) is not longer maintained and
+The [synapse LDAP3 auth provider](https://github.com/matrix-org/matrix-synapse-ldap3) is not longer maintained despite
-only handles on specific flow: validate credentials at login.
+saying so and only handles on specific flow: validate credentials at login.
 
 It does not:
 - Auto-provision user profiles
 - Integrate with Identity management
 - Integrate with Directory searches
+- Integrate with Profile data
 
 mxisd is a replacement and enhancement of it, offering coherent results in all areas, which the LDAP3 auth provider
 does not.
@@ -74,7 +84,7 @@ No.
 In its default configuration, mxisd does not talk to the central Identity server matrix.org to avoid leaking your private
 data and those of people you might know.
 
-mxisd [can be configured](features/identity.md#lookups) to talk to the central Identity servers if you wish.
+[You can configure it](features/identity.md#lookups) to talk to the central Identity servers if you wish.
 
 ### So mxisd is just a big hack! I don't want to use non-official features!
 mxisd primary concerns are your privacy and to always be compatible with the Matrix ecosystem and the Identity service API.  

docs/features/identity.md

@@ -1,7 +1,7 @@
 # Identity
 **WARNING**: This document is incomplete and can be misleading.
 
-Implementation of the [Unofficial Matrix Identity Service API](https://kamax.io/matrix/api/identity_service/unstable.html).
+Implementation of the [Identity Service API r0.1.0](https://matrix.org/docs/spec/identity_service/r0.1.0.html).
 
 ## Lookups
 If you would like to use the central matrix.org Identity server to ensure maximum discovery at the cost of potentially

docs/getting-started.md

@@ -6,8 +6,7 @@
 5. [Validate](#validate)
 6. [Next steps](#next-steps)
 
-Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups and
+Following these quick start instructions, you will have a basic setup that can perform recursive/federated lookups.  
-talk to the central Matrix.org Identity server.  
 This will be a good ground work for further integration with features and your existing Identity stores.
 
 ---
@@ -24,13 +23,17 @@ You will need:
 - Working Homeserver, ideally with working federation
 - Reverse proxy with regular TLS/SSL certificate (Let's encrypt) for your mxisd domain
 
-As synapse requires an HTTPS connection when talking to an Identity service, **a reverse proxy is required** as mxisd does
+If you use synapse:
-not support HTTPS listener at this time.
+- It requires an HTTPS connection when talking to an Identity service, **a reverse proxy is required** as mxisd does
+  not support HTTPS listener at this time.
+- HTTPS is hardcoded when talking to the Identity server. If your Identity server URL in your client is `https://matrix.example.org/`,
+  then you need to ensure `https://matrix.example.org/_matrix/identity/api/v1/...` will reach mxisd if called from the synapse host.
+  In doubt, test with `curl` or similar. 
 
-For maximum integration, it is best to have your Homeserver and mxisd reachable via the same hostname.
+For maximum integration, it is best to have your Homeserver and mxisd reachable via the same public hostname.
 
 Be aware of a [NAT/Reverse proxy gotcha](https://github.com/kamax-matrix/mxisd/wiki/Gotchas#nating) if you use the same
-hostname.
+host.
 
 The following Quick Start guide assumes you will host the Homeserver and mxisd under the same hostname.  
 If you would like a high-level view of the infrastructure and how each feature is integrated, see the
@@ -83,7 +86,7 @@ ProxyPass /_matrix/identity http://0.0.0.0:8090/_matrix/identity
 Typical configuration would look like:
 ```apache
 <VirtualHost *:443>
-    ServerName example.org
+    ServerName matrix.example.org
     
     ...
     
@@ -107,7 +110,7 @@ Typical configuration would look like:
 ```nginx
 server {
     listen 443 ssl;
-    server_name example.org;
+    server_name matrix.example.org;
     
     ...
     
@@ -130,17 +133,17 @@ Add your mxisd domain into the `homeserver.yaml` at `trusted_third_party_id_serv
 In a typical configuration, you would end up with something similar to:
 ```yaml
 trusted_third_party_id_servers:
-    - example.org
+    - matrix.example.org
 ```
-It is recommended to remove `matrix.org` and `vector.im` (or any other default entry) from your configuration so only
+It is **highly recommended** to remove `matrix.org` and `vector.im` (or any other default entry) from your configuration
-your own Identity server is authoritative for your HS.
+so only your own Identity server is authoritative for your HS.
 
 ## Validate
 **NOTE:** In case your homeserver has no working federation, step 5 will not happen. If step 4 took place, consider
 your installation validated.
 
-1. Log in using your Matrix client and set `https://example.org` as your Identity server URL, replacing `example.org` by
+1. Log in using your Matrix client and set `https://matrix.example.org` as your Identity server URL, replacing `matrix.example.org`
-the relevant hostname which you configured in your reverse proxy.
+by the relevant hostname which you configured in your reverse proxy.
 2. Create a new empty room. All further actions will take place in this room.
 3. Invite `mxisd-federation-test@kamax.io`
 4. The 3PID invite should be turned into a Matrix invite to `@mxisd-lookup-test:kamax.io`.

docs/stores/exec.md

@@ -39,7 +39,7 @@
 | [Authentication](../features/authentication.md) | Yes       |
 | [Directory](../features/directory.md)           | Yes       |
 | [Identity](../features/identity.md)             | Yes       |
-| [Profile](#profile)                             | Yes       |
+| [Profile](../features/profile.md)               | Yes       |
 
 This Identity Store lets you run arbitrary commands to handle the various requests in each support feature.  
 It is the most versatile Identity store of mxisd, allowing you to connect any kind of logic with any executable/script.
@@ -199,7 +199,7 @@ exec:
     DOMAIN: '{domain}'
 ```
 With Authentication enabled, run `/opt/mxisd-exec/auth.sh` when validating credentials, providing:
-- A single command-line argument to provide the `localoart` as username 
+- A single command-line argument to provide the `localpart` as username 
 - A plain text string with the password token for standard input, which will be replaced by the password to check
 - A single environment variable `DOMAIN` containing Matrix ID domain, if given
 
@@ -207,7 +207,7 @@ The command will use the default values for:
 - Success exit status of `0`
 - Failure exit status of `1`
 - Any other exit status considered as error
-- The standard output processing as not processed
+- Standard output will not be processed
 
 #### Advanced
 Given the fictional `placeholder` feature:

docs/stores/firebase.md

@@ -2,12 +2,12 @@
 https://firebase.google.com/
 
 ## Features
-|      Name      | Supported? |
+|                       Name                      | Supported |
-|----------------|------------|
+|-------------------------------------------------|-----------|
-| Authentication | Yes        |
+| [Authentication](../features/authentication.md) | Yes       |
-| Directory      | No         |
+| [Directory](../features/directory.md)           | No        |
-| Identity       | Yes        |
+| [Identity](../features/identity.md)             | Yes       |
-| Profile        | No         |
+| [Profile](../features/profile.md)               | No        |
 
 ## Requirements
 This backend requires a suitable Matrix client capable of performing Firebase authentication and passing the following

docs/stores/ldap.md

@@ -8,12 +8,12 @@
 For NetIQ, replace all the `ldap` prefix in the configuration by `netiq`.
 
 ## Features
-|      Name      | Supported? |
+|                       Name                      | Supported |
-|----------------|------------|
+|-------------------------------------------------|-----------|
-| Authentication | Yes        |
+| [Authentication](../features/authentication.md) | Yes       |
-| Directory      | Yes        |
+| [Directory](../features/directory.md)           | Yes       |
-| Identity       | Yes        |
+| [Identity](../features/identity.md)             | Yes       |
-| Profile        | Yes        |
+| [Profile](../features/profile.md)               | Yes       |
 
 ## Getting started
 ### Base
@@ -113,16 +113,18 @@ configuration item is needed to get started.
 - `ldap.identity.medium`: Namespace to overwrite generated queries from the list of attributes for each 3PID medium.
 
 ### Authentication
-No further configuration is needed to use the Authentication feature with LDAP once globally enabled and configured.
+After you have configured and enabled the [feature itself](../features/authentication.md), no further configuration is
+needed with this identity store to make it work.
 
 Profile auto-fill is enabled by default. It will use the `ldap.attribute.name` and `ldap.attribute.threepid` configuration
 options to get a lit of attributes to be used to build the user profile to pass on to synapse during authentication.
 
 #### Configuration
-- `ldap.auth.filter`: Specific user filter applied during identity search. Global filter is used if blank/not set.
+- `ldap.auth.filter`: Specific user filter applied during username search. Global filter is used if blank/not set.
 
 ### Directory
-No further configuration is needed to use the Directory feature with LDAP once globally enabled and configured.
+After you have configured and enabled the [feature itself](../features/directory.md), no further configuration is
+needed with this identity store to make it work.
 
 #### Configuration
 To set a specific filter applied during directory search, use `ldap.directory.filter`

docs/stores/sql.md

@@ -6,12 +6,12 @@
 - SQLite
 
 ## Features
-|      Name      | Supported? |
+|                       Name                      | Supported |
-|----------------|------------|
+|-------------------------------------------------|-----------|
-| Authentication | No         |
+| [Authentication](../features/authentication.md) | No        |
-| Directory      | Yes        |
+| [Directory](../features/directory.md)           | Yes       |
-| Identity       | Yes        |
+| [Identity](../features/identity.md)             | Yes       |
-| Profile        | Yes        |
+| [Profile](../features/profile.md)               | Yes       |
 
 Due to the implementation complexity of supporting arbitrary hashing/encoding mechanisms or auth flow, Authentication
 will be out of scope of SQL Identity stores and should be done via one of the other identity stores, typically

docs/stores/synapse.md

@@ -2,12 +2,12 @@
 Synapse's Database itself can be used as an Identity store.
 
 ## Features
-|      Name      | Supported? |
+|                       Name                      | Supported |
-|----------------|------------|
+|-------------------------------------------------|-----------|
-| Authentication | No         |
+| [Authentication](../features/authentication.md) | No        |
-| Directory      | Yes        |
+| [Directory](../features/directory.md)           | Yes       |
-| Identity       | Yes        |
+| [Identity](../features/identity.md)             | Yes       |
-| Profile        | Yes        |
+| [Profile](../features/profile.md)               | Yes       |
 
 Authentication is done by Synapse itself.
 

docs/stores/wordpress.md

@@ -5,12 +5,12 @@ Two types of connections are required for full support:
 - Direct SQL access
 
 ## Features
-|      Name      | Supported? |
+|                       Name                      | Supported |
-|----------------|------------|
+|-------------------------------------------------|-----------|
-| Authentication | Yes        |
+| [Authentication](../features/authentication.md) | Yes       |
-| Directory      | Yes        |
+| [Directory](../features/directory.md)           | Yes       |
-| Identity       | Yes        |
+| [Identity](../features/identity.md)             | Yes       |
-| Profile        | No         |
+| [Profile](../features/profile.md)               | No        |
 
 ## Requirements
 - [Wordpress](https://wordpress.org/download/) >= 4.4

docs/threepids/notification/sendgrid-handler.md

@@ -26,13 +26,7 @@ notification:
             html: <Path to file containing the HTML part of the email. Do not set to not use one>
         session:
           validation:
-            local:
+            subject: <Subject of the email notification sent for 3PID sessions>
-              subject: <Subject of the email notification sent for local 3PID sessions>
-              body:
-                text: <Path to file containing the raw text part of the email. Do not set to not use one>
-                html: <Path to file containing the HTML part of the email. Do not set to not use one>
-            remote:
-              subject: <Subject of the email notification sent for remote 3PID sessions>
             body:
               text: <Path to file containing the raw text part of the email. Do not set to not use one>
               html: <Path to file containing the HTML part of the email. Do not set to not use one>

docs/threepids/notification/template-generator.md

@@ -18,9 +18,7 @@ threepid:
         template:
           invite: '/path/to/invite-template.eml'
           session:
-            validation:
+            validation: '/path/to/validate-template.eml'
-              local: '/path/to/validate-local-template.eml'
-              remote: '/path/to/validate-remote-template.eml'
             unbind:
               frandulent: '/path/to/unbind-fraudulent-template.eml'
           generic:

src/main/java/io/kamax/mxisd/HttpMxisd.java

@@ -53,6 +53,7 @@ public class HttpMxisd {
     public void start() {
         m.start();
 
+        HttpHandler helloHandler = SaneHandler.around(new HelloHandler());
         HttpHandler asNotFoundHandler = SaneHandler.around(new AsNotFoundHandler(m.getAs()));
         HttpHandler asTxnHandler = SaneHandler.around(new AsTransactionHandler(m.getAs()));
         HttpHandler storeInvHandler = SaneHandler.around(new StoreInviteHandler(m.getConfig().getServer(), m.getInvitationManager(), m.getKeyManager()));
@@ -79,7 +80,8 @@ public class HttpMxisd {
                 .get(EphemeralKeyIsValidHandler.Path, SaneHandler.around(new EphemeralKeyIsValidHandler()))
 
                 // Identity endpoints
-                .get(HelloHandler.Path, SaneHandler.around(new HelloHandler()))
+                .get(HelloHandler.Path, helloHandler)
+                .get(HelloHandler.Path + "/", helloHandler) // Be lax with possibly trailing slash
                 .get(SingleLookupHandler.Path, SaneHandler.around(new SingleLookupHandler(m.getIdentity(), m.getSign())))
                 .post(BulkLookupHandler.Path, SaneHandler.around(new BulkLookupHandler(m.getIdentity())))
                 .post(StoreInviteHandler.Path, storeInvHandler)
@@ -109,7 +111,6 @@ public class HttpMxisd {
 
     public void stop() {
         httpSrv.stop();
-
         m.stop();
     }
 

src/main/java/io/kamax/mxisd/Mxisd.java

@@ -40,6 +40,7 @@ import io.kamax.mxisd.lookup.provider.BridgeFetcher;
 import io.kamax.mxisd.lookup.provider.RemoteIdentityServerFetcher;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.lookup.strategy.RecursivePriorityLookupStrategy;
+import io.kamax.mxisd.matrix.IdentityServerUtils;
 import io.kamax.mxisd.notification.NotificationHandlerSupplier;
 import io.kamax.mxisd.notification.NotificationHandlers;
 import io.kamax.mxisd.notification.NotificationManager;
@@ -55,37 +56,38 @@ import java.util.ServiceLoader;
 
 public class Mxisd {
 
-    protected MxisdConfig cfg;
+    private MxisdConfig cfg;
 
-    protected CloseableHttpClient httpClient;
+    private CloseableHttpClient httpClient;
-    protected IRemoteIdentityServerFetcher srvFetcher;
+    private IRemoteIdentityServerFetcher srvFetcher;
 
-    protected IStorage store;
+    private IStorage store;
 
-    protected KeyManager keyMgr;
+    private KeyManager keyMgr;
-    protected SignatureManager signMgr;
+    private SignatureManager signMgr;
 
     // Features
-    protected AuthManager authMgr;
+    private AuthManager authMgr;
-    protected DirectoryManager dirMgr;
+    private DirectoryManager dirMgr;
-    protected LookupStrategy idStrategy;
+    private LookupStrategy idStrategy;
-    protected InvitationManager invMgr;
+    private InvitationManager invMgr;
-    protected ProfileManager pMgr;
+    private ProfileManager pMgr;
-    protected AppSvcManager asHander;
+    private AppSvcManager asHander;
-    protected SessionManager sessMgr;
+    private SessionManager sessMgr;
-    protected NotificationManager notifMgr;
+    private NotificationManager notifMgr;
 
     public Mxisd(MxisdConfig cfg) {
         this.cfg = cfg.build();
     }
 
-    protected void build() {
+    private void build() {
         httpClient = HttpClients.custom()
                 .setUserAgent("mxisd")
                 .setMaxConnPerRoute(Integer.MAX_VALUE)
                 .setMaxConnTotal(Integer.MAX_VALUE)
                 .build();
 
+        IdentityServerUtils.setHttpClient(httpClient);
         srvFetcher = new RemoteIdentityServerFetcher(httpClient);
 
         store = new OrmLiteSqlStorage(cfg);

src/main/java/io/kamax/mxisd/MxisdStandaloneExec.java

@@ -23,6 +23,8 @@ package io.kamax.mxisd;
 import io.kamax.mxisd.config.MxisdConfig;
 import io.kamax.mxisd.config.YamlConfigLoader;
 import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 import java.io.IOException;
 import java.util.Arrays;
@@ -31,7 +33,10 @@ import java.util.Objects;
 
 public class MxisdStandaloneExec {
 
+    private static final Logger log = LoggerFactory.getLogger("");
+
     public static void main(String[] args) throws IOException {
+        log.info("------------- mxisd starting -------------");
         MxisdConfig cfg = null;
 
         Iterator<String> argsIt = Arrays.asList(args).iterator();
@@ -40,9 +45,8 @@ public class MxisdStandaloneExec {
             if (StringUtils.equals("-c", arg)) {
                 String cfgFile = argsIt.next();
                 cfg = YamlConfigLoader.loadFromFile(cfgFile);
-                System.out.println("Loaded configuration from " + cfgFile);
             } else {
-                System.out.println("Invalid argument: " + arg);
+                log.info("Invalid argument: {}", arg);
                 System.exit(1);
             }
         }
@@ -55,11 +59,11 @@ public class MxisdStandaloneExec {
             HttpMxisd mxisd = new HttpMxisd(cfg);
             Runtime.getRuntime().addShutdownHook(new Thread(() -> {
                 mxisd.stop();
-                System.out.println("------------- mxisd stopped -------------");
+                log.info("------------- mxisd stopped -------------");
             }));
             mxisd.start();
 
-            System.out.println("------------- mxisd started -------------");
+            log.info("------------- mxisd started -------------");
         } catch (Throwable t) {
             t.printStackTrace();
             System.exit(1);

src/main/java/io/kamax/mxisd/config/DirectoryConfig.java

@@ -64,8 +64,8 @@ public class DirectoryConfig {
     public void build() {
         log.info("--- Directory config ---");
         log.info("Exclude:");
-        log.info("\tHomeserver: {}", getExclude().getHomeserver());
+        log.info("  Homeserver: {}", getExclude().getHomeserver());
-        log.info("\t3PID: {}", getExclude().getThreepid());
+        log.info("  3PID: {}", getExclude().getThreepid());
     }
 
 }

src/main/java/io/kamax/mxisd/config/YamlConfigLoader.java

@@ -21,6 +21,8 @@
 package io.kamax.mxisd.config;
 
 import io.kamax.matrix.json.GsonUtil;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 import org.yaml.snakeyaml.Yaml;
 import org.yaml.snakeyaml.constructor.Constructor;
 import org.yaml.snakeyaml.representer.Representer;
@@ -32,21 +34,29 @@ import java.util.Optional;
 
 public class YamlConfigLoader {
 
+    private static final Logger log = LoggerFactory.getLogger(YamlConfigLoader.class);
+
     public static MxisdConfig loadFromFile(String path) throws IOException {
+        log.debug("Reading config from {}", path);
         Representer rep = new Representer();
         rep.getPropertyUtils().setAllowReadOnlyProperties(true);
         rep.getPropertyUtils().setSkipMissingProperties(true);
         Yaml yaml = new Yaml(new Constructor(MxisdConfig.class), rep);
         try (FileInputStream is = new FileInputStream(path)) {
             Object o = yaml.load(is);
-            return GsonUtil.get().fromJson(GsonUtil.get().toJson(o), MxisdConfig.class);
+            log.debug("Read config in memory from {}", path);
+            MxisdConfig cfg = GsonUtil.get().fromJson(GsonUtil.get().toJson(o), MxisdConfig.class);
+            log.info("Loaded config from {}", path);
+            return cfg;
         }
     }
 
     public static Optional<MxisdConfig> tryLoadFromFile(String path) {
+        log.debug("Attempting to read config from {}", path);
         try {
             return Optional.of(loadFromFile(path));
         } catch (FileNotFoundException e) {
+            log.info("No config file at {}", path);
             return Optional.empty();
         } catch (IOException e) {
             throw new RuntimeException(e);

src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java

@@ -421,9 +421,9 @@ public abstract class LdapConfig {
         log.info("Port: {}", connection.getPort());
         log.info("TLS: {}", connection.isTls());
         log.info("Bind DN: {}", connection.getBindDn());
-        log.info("Base DNs: {}");
+        log.info("Base DNs:");
         for (String baseDN : connection.getBaseDNs()) {
-            log.info("\t- {}", baseDN);
+            log.info("  - {}", baseDN);
         }
 
         log.info("Attribute: {}", GsonUtil.get().toJson(attribute));

src/main/java/io/kamax/mxisd/config/threepid/connector/EmailSendGridConfig.java

@@ -115,28 +115,6 @@ public class EmailSendGridConfig {
 
     public static class Templates {
 
-        public static class TemplateSessionValidation {
-
-            private EmailTemplate local = new EmailTemplate();
-            private EmailTemplate remote = new EmailTemplate();
-
-            public EmailTemplate getLocal() {
-                return local;
-            }
-
-            public void setLocal(EmailTemplate local) {
-                this.local = local;
-            }
-
-            public EmailTemplate getRemote() {
-                return remote;
-            }
-
-            public void setRemote(EmailTemplate remote) {
-                this.remote = remote;
-            }
-        }
-
         public static class TemplateSessionUnbind {
 
             private EmailTemplate fraudulent = new EmailTemplate();
@@ -153,14 +131,14 @@ public class EmailSendGridConfig {
 
         public static class TemplateSession {
 
-            private TemplateSessionValidation validation = new TemplateSessionValidation();
+            private EmailTemplate validation = new EmailTemplate();
             private TemplateSessionUnbind unbind = new TemplateSessionUnbind();
 
-            public TemplateSessionValidation getValidation() {
+            public EmailTemplate getValidation() {
                 return validation;
             }
 
-            public void setValidation(TemplateSessionValidation validation) {
+            public void setValidation(EmailTemplate validation) {
                 this.validation = validation;
             }
 

src/main/java/io/kamax/mxisd/config/threepid/medium/EmailTemplateConfig.java

@@ -30,17 +30,17 @@ public class EmailTemplateConfig extends GenericTemplateConfig {
     public EmailTemplateConfig() {
         setInvite("classpath:/threepids/email/invite-template.eml");
         getGeneric().put("matrixId", "classpath:/threepids/email/mxid-template.eml");
-        getSession().getValidation().setLocal("classpath:/threepids/email/validate-local-template.eml");
+        getSession().setValidation("classpath:/threepids/email/validate-template.eml");
-        getSession().getValidation().setRemote("classpath:/threepids/email/validate-remote-template.eml");
         getSession().getUnbind().setFraudulent("classpath:/threepids/email/unbind-fraudulent.eml");
     }
 
     public EmailTemplateConfig build() {
         log.info("--- E-mail Generator templates config ---");
         log.info("Invite: {}", getName(getInvite()));
-        log.info("Session validation:");
+        log.info("Session:");
-        log.info("\tLocal: {}", getName(getSession().getValidation().getLocal()));
+        log.info("  Validation: {}", getSession().getValidation());
-        log.info("\tRemote: {}", getName(getSession().getValidation().getRemote()));
+        log.info("  Unbind:");
+        log.info("    Fraudulent: {}", getSession().getUnbind().getFraudulent());
 
         return this;
     }

src/main/java/io/kamax/mxisd/config/threepid/medium/GenericTemplateConfig.java

@@ -39,29 +39,6 @@ public class GenericTemplateConfig {
 
     public static class Session {
 
-        public static class SessionValidation {
-
-            private String local;
-            private String remote;
-
-            public String getLocal() {
-                return local;
-            }
-
-            public void setLocal(String local) {
-                this.local = local;
-            }
-
-            public String getRemote() {
-                return remote;
-            }
-
-            public void setRemote(String remote) {
-                this.remote = remote;
-            }
-
-        }
-
         public static class SessionUnbind {
 
             private String fraudulent;
@@ -76,14 +53,14 @@ public class GenericTemplateConfig {
 
         }
 
-        private SessionValidation validation = new SessionValidation();
+        private String validation;
         private SessionUnbind unbind = new SessionUnbind();
 
-        public SessionValidation getValidation() {
+        public String getValidation() {
             return validation;
         }
 
-        public void setValidation(SessionValidation validation) {
+        public void setValidation(String validation) {
             this.validation = validation;
         }
 

src/main/java/io/kamax/mxisd/config/threepid/medium/PhoneSmsTemplateConfig.java

@@ -29,18 +29,17 @@ public class PhoneSmsTemplateConfig extends GenericTemplateConfig {
 
     public PhoneSmsTemplateConfig() {
         setInvite("classpath:/threepids/sms/invite-template.txt");
-        getGeneric().put("matrixId", "classpath:/threepids/email/mxid-template.eml");
+        getSession().setValidation("classpath:/threepids/sms/validate-template.txt");
-        getSession().getValidation().setLocal("classpath:/threepids/sms/validate-local-template.txt");
-        getSession().getValidation().setRemote("classpath:/threepids/sms/validate-remote-template.txt");
         getSession().getUnbind().setFraudulent("classpath:/threepids/sms/unbind-fraudulent.txt");
     }
 
     public PhoneSmsTemplateConfig build() {
         log.info("--- SMS Generator templates config ---");
         log.info("Invite: {}", getName(getInvite()));
-        log.info("Session validation:");
+        log.info("Session:");
-        log.info("\tLocal: {}", getName(getSession().getValidation().getLocal()));
+        log.info("  Validation: {}", getSession().getValidation());
-        log.info("\tRemote: {}", getName(getSession().getValidation().getRemote()));
+        log.info("  Unbind:");
+        log.info("    Fraudulent: {}", getSession().getUnbind().getFraudulent());
 
         return this;
     }

src/main/java/io/kamax/mxisd/config/threepid/notification/NotificationConfig.java

@@ -61,7 +61,7 @@ public class NotificationConfig {
     public void build() {
         log.info("--- Notification config ---");
         log.info("Handlers:");
-        handler.forEach((k, v) -> log.info("\t{}: {}", k, v));
+        handler.forEach((k, v) -> log.info("  {}: {}", k, v));
     }
 
 }

src/main/java/io/kamax/mxisd/directory/DirectoryManager.java

@@ -62,7 +62,7 @@ public class DirectoryManager {
         this.providers = new ArrayList<>(providers);
 
         log.info("Directory providers:");
-        this.providers.forEach(p -> log.info("\t- {}", p.getClass().getName()));
+        this.providers.forEach(p -> log.info("  - {}", p.getClass().getName()));
     }
 
     public UserDirectorySearchResult search(URI target, String accessToken, String query) {

src/main/java/io/kamax/mxisd/lookup/provider/RemoteIdentityServerFetcher.java

@@ -25,6 +25,7 @@ import com.google.gson.JsonObject;
 import com.google.gson.JsonParseException;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.exception.InvalidResponseJsonException;
+import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.http.io.identity.ClientBulkLookupRequest;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
@@ -73,7 +74,7 @@ public class RemoteIdentityServerFetcher implements IRemoteIdentityServerFetcher
 
         try {
             URIBuilder b = new URIBuilder(remote);
-            b.setPath("/_matrix/identity/api/v1/lookup");
+            b.setPath(IsAPIv1.Base + "/lookup");
             b.addParameter("medium", request.getType());
             b.addParameter("address", request.getThreePid());
             HttpGet req = new HttpGet(b.build());
@@ -116,7 +117,7 @@ public class RemoteIdentityServerFetcher implements IRemoteIdentityServerFetcher
         ClientBulkLookupRequest mappingRequest = new ClientBulkLookupRequest();
         mappingRequest.setMappings(mappings);
 
-        String url = remote + "/_matrix/identity/api/v1/bulk_lookup";
+        String url = remote + IsAPIv1.Base + "/bulk_lookup";
         try {
             HttpPost request = RestClientUtils.post(url, mappingRequest);
             try (CloseableHttpResponse response = client.execute(request)) {

src/main/java/io/kamax/mxisd/lookup/strategy/RecursivePriorityLookupStrategy.java

@@ -57,7 +57,7 @@ public class RecursivePriorityLookupStrategy implements LookupStrategy {
 
         try {
             log.info("Found {} providers", providers.size());
-            providers.forEach(p -> log.info("\t- {}", p.getClass().getName()));
+            providers.forEach(p -> log.info("  - {}", p.getClass().getName()));
             providers.sort((o1, o2) -> Integer.compare(o2.getPriority(), o1.getPriority()));
 
             log.info("Recursive lookup enabled: {}", cfg.getRecursive().isEnabled());

src/main/java/io/kamax/mxisd/matrix/IdentityServerUtils.java

@@ -1,17 +1,42 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2017 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
 package io.kamax.mxisd.matrix;
 
 import com.google.gson.JsonElement;
 import com.google.gson.JsonParseException;
 import com.google.gson.JsonParser;
+import io.kamax.mxisd.http.IsAPIv1;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.xbill.DNS.*;
 
 import java.io.IOException;
-import java.net.HttpURLConnection;
 import java.net.MalformedURLException;
+import java.net.URI;
 import java.net.URL;
 import java.nio.charset.StandardCharsets;
 import java.util.ArrayList;
@@ -20,31 +45,41 @@ import java.util.List;
 import java.util.Optional;
 
 // FIXME placeholder, this must go in matrix-java-sdk for 1.0
+// FIXME this class is just a mistake and should never have happened. Make sure to get rid of for v2.x
 public class IdentityServerUtils {
 
     private static Logger log = LoggerFactory.getLogger(IdentityServerUtils.class);
     private static JsonParser parser = new JsonParser();
 
+    private static CloseableHttpClient client;
+
+    public static void setHttpClient(CloseableHttpClient client) {
+        IdentityServerUtils.client = client;
+    }
+
     public static boolean isUsable(String remote) {
         if (StringUtils.isBlank(remote)) {
+            log.info("IS URL is blank, not usable");
             return false;
         }
 
-        try {
+        HttpGet req = new HttpGet(URI.create(remote + IsAPIv1.Base));
-            // FIXME use Apache HTTP client
+        req.setConfig(RequestConfig.custom()
-            HttpURLConnection rootSrvConn = (HttpURLConnection) new URL(remote + "/_matrix/identity/api/v1/").openConnection();
+                .setConnectTimeout(2000)
-            // TODO turn this into a configuration property
+                .setConnectionRequestTimeout(2000)
-            rootSrvConn.setConnectTimeout(2000);
+                .build()
+        );
 
-            int status = rootSrvConn.getResponseCode();
+        try (CloseableHttpResponse res = client.execute(req)) {
+            int status = res.getStatusLine().getStatusCode();
             if (status != 200) {
                 log.info("Usability of {} as Identity server: answer status: {}", remote, status);
                 return false;
             }
 
-            JsonElement el = parser.parse(IOUtils.toString(rootSrvConn.getInputStream(), StandardCharsets.UTF_8));
+            JsonElement el = parser.parse(IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8));
             if (!el.isJsonObject()) {
-                log.debug("IS {} did not send back a JSON object for single 3PID lookup");
+                log.debug("IS {} did not send back an empty JSON object as per spec, not a valid IS");
                 return false;
             }
 

src/main/java/io/kamax/mxisd/notification/NotificationHandler.java

@@ -37,8 +37,6 @@ public interface NotificationHandler {
 
     void sendForValidation(IThreePidSession session);
 
-    void sendForRemoteValidation(IThreePidSession session);
-
     void sendForFraudulentUnbind(ThreePid tpid);
 
 }

src/main/java/io/kamax/mxisd/notification/NotificationManager.java

@@ -78,10 +78,6 @@ public class NotificationManager {
         ensureMedium(session.getThreePid().getMedium()).sendForValidation(session);
     }
 
-    public void sendForRemoteValidation(IThreePidSession session) {
-        ensureMedium(session.getThreePid().getMedium()).sendForRemoteValidation(session);
-    }
-
     public void sendForFraudulentUnbind(ThreePid tpid) throws NotImplementedException {
         ensureMedium(tpid.getMedium()).sendForFraudulentUnbind(tpid);
     }

src/main/java/io/kamax/mxisd/profile/ProfileManager.java

@@ -58,7 +58,7 @@ public class ProfileManager {
         this.providers = new ArrayList<>(providers);
 
         log.info("Profile Providers:");
-        providers.forEach(p -> log.info("\t- {}", p.getClass().getSimpleName()));
+        providers.forEach(p -> log.info("  - {}", p.getClass().getSimpleName()));
     }
 
     public <T> List<T> getList(Function<ProfileProvider, List<T>> function) {

src/main/java/io/kamax/mxisd/threepid/generator/GenericTemplateNotificationGenerator.java

@@ -74,13 +74,7 @@ public abstract class GenericTemplateNotificationGenerator extends PlaceholderNo
     @Override
     public String getForValidation(IThreePidSession session) {
         log.info("Generating notification content for 3PID Session validation");
-        return populateForValidation(session, getTemplateContent(cfg.getSession().getValidation().getLocal()));
+        return populateForValidation(session, getTemplateContent(cfg.getSession().getValidation()));
-    }
-
-    @Override
-    public String getForRemoteValidation(IThreePidSession session) {
-        log.info("Generating notification content for remote-only 3PID session");
-        return populateForRemoteValidation(session, getTemplateContent(cfg.getSession().getValidation().getRemote()));
     }
 
     @Override

src/main/java/io/kamax/mxisd/threepid/generator/NotificationGenerator.java

@@ -37,8 +37,6 @@ public interface NotificationGenerator {
 
     String getForValidation(IThreePidSession session);
 
-    String getForRemoteValidation(IThreePidSession session);
-
     String getForFraudulentUnbind(ThreePid tpid);
 
 }

src/main/java/io/kamax/mxisd/threepid/notification/GenericNotificationHandler.java

@@ -72,11 +72,6 @@ public abstract class GenericNotificationHandler<A extends ThreePidConnector, B
         send(connector, session.getThreePid().getAddress(), generator.getForValidation(session));
     }
 
-    @Override
-    public void sendForRemoteValidation(IThreePidSession session) {
-        send(connector, session.getThreePid().getAddress(), generator.getForRemoteValidation(session));
-    }
-
     @Override
     public void sendForFraudulentUnbind(ThreePid tpid) {
         send(connector, tpid.getAddress(), generator.getForFraudulentUnbind(tpid));

src/main/java/io/kamax/mxisd/threepid/notification/email/EmailSendGridNotificationHandler.java

@@ -108,7 +108,7 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
 
     @Override
     public void sendForValidation(IThreePidSession session) {
-        EmailTemplate template = cfg.getTemplates().getSession().getValidation().getLocal();
+        EmailTemplate template = cfg.getTemplates().getSession().getValidation();
         Email email = getEmail();
         email.setSubject(populateForValidation(session, template.getSubject()));
         email.setText(populateForValidation(session, getFromFile(template.getBody().getText())));
@@ -117,17 +117,6 @@ public class EmailSendGridNotificationHandler extends PlaceholderNotificationGen
         send(session.getThreePid().getAddress(), email);
     }
 
-    @Override
-    public void sendForRemoteValidation(IThreePidSession session) {
-        EmailTemplate template = cfg.getTemplates().getSession().getValidation().getRemote();
-        Email email = getEmail();
-        email.setSubject(populateForRemoteValidation(session, template.getSubject()));
-        email.setText(populateForRemoteValidation(session, getFromFile(template.getBody().getText())));
-        email.setHtml(populateForRemoteValidation(session, getFromFile(template.getBody().getHtml())));
-
-        send(session.getThreePid().getAddress(), email);
-    }
-
     @Override
     public void sendForFraudulentUnbind(ThreePid tpid) {
         EmailTemplate template = cfg.getTemplates().getSession().getUnbind().getFraudulent();