Add support for multiple Base DNs in LDAP Identity Store (Fix #104)

- Process transactions async with completion parking
- Detect transactions deduplication

build.gradle

@@ -139,6 +139,7 @@ dependencies {
 
     testCompile 'junit:junit:4.12'
     testCompile 'com.github.tomakehurst:wiremock:2.8.0'
+    testCompile 'com.unboundid:unboundid-ldapsdk:4.0.9'
 }
 
 springBoot {

docs/features/authentication.md

@@ -74,7 +74,15 @@ See your Identity store [documentation](../stores/README.md) on how to enable th
 
 
 ## Advanced
-The Authentication feature allows users to login to their Homeserver by using their 3PIDs in a configured Identity store.
+The Authentication feature allows users to:
+- Rewrite usernames matching a pattern to be mapped to another username via a 3PID.
+- login to their Homeserver by using their 3PIDs in a configured Identity store.
+
+This feature also allows to work around the following issues:
+- Lowercase all usernames for synapse, allowing case-insensitive login
+- Unable to login on synapse if username is numerical
+- Any generic transformation of username prior to sending to synapse, bypassing the restriction that password providers
+cannot change the localpart being authenticated.
 
 ### Overview
 This is performed by intercepting the Homeserver endpoint `/_matrix/client/r0/login` as depicted below:
@@ -109,10 +117,10 @@ Steps of user authentication using a 3PID:
 4. The response from the Homeserver is sent back to the client, believing it was the HS which directly answered.
 
 ### Requirements
-- [Basic Authentication configured and working](#basic)
-- Reverse proxy setup
-- Homeserver
 - Compatible [Identity store](../stores/README.md)
+- [Basic Authentication configured and working](#basic)
+- Client and Homeserver using the [C2S API r0.4.x](https://matrix.org/docs/spec/client_server/r0.4.0.html) or later
+- Reverse proxy setup
 
 ### Configuration
 #### Reverse Proxy
@@ -153,3 +161,40 @@ In case the hostname is the same as your Matrix domain and `server.name` is not
 `matrix.domain` and will still probably have the correct value.
 
 `value` is the base internal URL of the Homeserver, without any `/_matrix/..` or trailing `/`.
+
+#### Username rewrite
+In mxisd config:
+```yaml
+auth:
+  rewrite:
+    user:
+      rules:
+        - regex: <your regexp>
+          medium: 'your.custom.medium.type'
+```
+`rules` takes a list of rules. Rules have two properties:
+- `regexp`: The regex pattern to match. This **MUST** match the full string. See [Java regex](https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html) for syntax.
+- `medium`: Custom 3PID type that will be used in the 3PID lookup. This can be anything you want and needs to be supported
+by your Identity store config and/or code.
+
+Rules are matched in listed order.
+
+Common regexp patterns:
+- Numerical usernames: `[0-9]+`
+
+##### LDAP Example
+If your users use their numerical employee IDs, which cannot be used with synapse, you can make it work with (relevant config only):
+```yaml
+auth:
+  rewrite:
+    user:
+      rules:
+        - regex: '[0-9]+'
+          medium: 'kmx.employee.id'
+          
+ldap:
+  attribute:
+    threepid:
+      kmx.employee.id:
+        - 'ldapAttributeForEmployeeId'
+```

docs/features/experimental/profile.md

@@ -1,12 +1,16 @@
-# Profile enhancement
-**WARNING**: Alpha feature, not officially supported. Do not use.
+# Profile
+**WARNING**: The following sub-features are considered experimental and not officially supported. Use at your own peril.
 
-This feature allows to enhance a profile query with more info than just Matrix ID and Display name, allowing for custom
-applications to retrieve custom data not currently provided by synapse, per example.
+## Public Profile enhancement
+This feature allows to enhance a public profile query with more info than just Matrix ID and Display name, allowing for
+custom applications to retrieve custom data not currently provided by synapse, per example.
 
-## Configuration
-### Reverse proxy
-#### Apache
+**WARNING**: This information can be queried without authentication as per the specification. Do not enable unless in a
+controlled environment.
+
+### Configuration
+#### Reverse proxy
+##### Apache
 ```apache
 ProxyPassMatch "^/_matrix/client/r0/profile/([^/]+)$" "http://127.0.0.1:8090/_matrix/client/r0/profile/$1"
 ```

docs/features/profile.md

@@ -0,0 +1,10 @@
+# Profile
+The profile feature does not do anything on its own and acts as a support feature for others, allowing to retrieve
+information about a user based on its Matrix ID by querying enabled [Identity stores](../stores/README.md).
+
+Currently supported:
+- Display name
+- 3PIDs
+- Roles/Groups
+
+Experimental sub-features are also available. See [the dedicated document](experimental/profile.md).

docs/stores/ldap.md

@@ -24,10 +24,13 @@ ldap.connection.host: 'ldapHostnameOrIp'
 ldap.connection.port: 389
 ldap.connection.bindDn: 'CN=My Mxisd User,OU=Users,DC=example,DC=org'
 ldap.connection.bindPassword: 'TheUserPassword'
-ldap.connection.baseDn: 'OU=Users,DC=example,DC=org'
+ldap.connection.baseDNs:
+  - 'OU=Users,DC=example,DC=org'
 ```
 These are standard LDAP connection configuration. mxisd will try to connect on port default port 389 without encryption.
 
+If you would like to use several Base DNs, simply add more entries under `baseDNs`.
+
 ### TLS/SSL connection
 If you would like to use a TLS/SSL connection, use the following configuration options (STARTLS not supported):
 ```yaml

docs/stores/rest.md

@@ -3,38 +3,45 @@ The REST backend allows you to query identity data in existing webapps, like:
 - Forums (phpBB, Discourse, etc.)
 - Custom Identity stores (Keycloak, ...)
 - CRMs (Wordpress, ...)
-- self-hosted clouds (Nextcloud, ownCloud, ...)
+- Self-hosted clouds (Nextcloud, ownCloud, ...)
 
-To integrate this backend with your webapp, you will need to implement three specific REST endpoints detailed below.
+To integrate this backend with your webapp, you will need to implement the REST endpoints described below.
 
 ## Features
-|      Name      | Supported? |
-|----------------|------------|
-| Authentication | Yes        |
-| Directory      | Yes        |
-| Identity       | Yes        |
-| Profile        | No         |
+| Name                                            | Supported? |
+|-------------------------------------------------|------------|
+| [Authentication](../features/authentication.md) | Yes        |
+| [Directory](../features/directory.md)           | Yes        |
+| [Identity](../features/identity.md)             | Yes        |
+| [Profile](../features/profile.md)               | Yes        |
 
 ## Configuration
-| Key                              | Default                                        | Description                                          |
-|----------------------------------|------------------------------------------------|------------------------------------------------------|
-| `rest.enabled`                   | `false`                                        | Globally enable/disable the REST backend             |
-| `rest.host`                      | *None*                                        | Default base URL to use for the different endpoints. |
-| `rest.endpoints.auth`            | `/_mxisd/backend/api/v1/auth/login`            | Validate credentials and get user profile            |
-| `rest.endpoints.directory`       | `/_mxisd/backend/api/v1/directory/user/search` | Search for users by arbitrary input                  |
-| `rest.endpoints.identity.single` | `/_mxisd/backend/api/v1/identity/single`       | Endpoint to query a single 3PID                      |
-| `rest.endpoints.identity.bulk`   | `/_mxisd/backend/api/v1/identity/bulk`         | Endpoint to query a list of 3PID                     |
+| Key                                  | Default                                        | Description                                          |
+|--------------------------------------|------------------------------------------------|------------------------------------------------------|
+| `rest.enabled`                       | `false`                                        | Globally enable/disable the REST backend             |
+| `rest.host`                          | *None*                                         | Default base URL to use for the different endpoints. |
+| `rest.endpoints.auth`                | `/_mxisd/backend/api/v1/auth/login`            | Validate credentials and get user profile            |
+| `rest.endpoints.directory`           | `/_mxisd/backend/api/v1/directory/user/search` | Search for users by arbitrary input                  |
+| `rest.endpoints.identity.single`     | `/_mxisd/backend/api/v1/identity/single`       | Endpoint to query a single 3PID                      |
+| `rest.endpoints.identity.bulk`       | `/_mxisd/backend/api/v1/identity/bulk`         | Endpoint to query a list of 3PID                     |
+| `rest.endpoints.profile.displayName` | `/_mxisd/backend/api/v1/profile/displayName`   | Query the display name for a Matrix ID
+| `rest.endpoints.profile.threepids`   | `/_mxisd/backend/api/v1/profile/threepids`     | Query the 3PIDs for a Matrix ID
+| `rest.endpoints.profile.roles`       | `/_mxisd/backend/api/v1/profile/roles`         | Query the Roles for a Matrix ID
 
 Endpoint values can handle two formats:
 - URL Path starting with `/` that gets happened to the `rest.host`
 - Full URL, if you want each endpoint to go to a specific server/protocol/port
 
+If an endpoint value is configured as an empty string, it will disable that specific feature, essentially bypassing the
+Identity store for that specific query.
+
 `rest.host` is mandatory if at least one endpoint is not a full URL.
 
 ## Endpoints
 ### Authentication
-HTTP method: `POST`  
-Content-type: JSON UTF-8
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
   
 #### Request Body
 ```json
@@ -87,8 +94,9 @@ If the authentication succeed:
 ```
 
 ### Directory
-HTTP method: `POST`
-Content-type: JSON UTF-8
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
 
 #### Request Body
 ```json
@@ -113,7 +121,7 @@ If users found:
       "user_id": "UserIdLocalpart"
     },
     {
-      ...
+      "...": "..."
     }
   ]
 }
@@ -129,10 +137,11 @@ If no user found:
 
 ### Identity
 #### Single 3PID lookup
-HTTP method: `POST`  
-Content-type: JSON UTF-8  
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
   
-#### Request Body
+##### Request Body
 ```json
 {
   "lookup": {
@@ -142,7 +151,7 @@ Content-type: JSON UTF-8
 }
 ```
 
-#### Response Body
+##### Response Body
 If a match was found:
 - `lookup.id.type` supported values: `localpart`, `mxid`
 ```json
@@ -164,10 +173,11 @@ If no match was found:
 ```
 
 #### Bulk 3PID lookup
-HTTP method: `POST`  
-Content-type: JSON UTF-8  
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
   
-#### Request Body
+##### Request Body
 ```json
 {
   "lookup": [
@@ -183,7 +193,7 @@ Content-type: JSON UTF-8
 }
 ```
 
-#### Response Body
+##### Response Body
 For all entries where a match was found:
 - `lookup[].id.type` supported values: `localpart`, `mxid`
 ```json
@@ -215,3 +225,46 @@ If no match was found:
   "lookup": []
 }
 ```
+
+### Profile
+#### Request Body
+For all requests, the values are the same:
+- Method: `POST`
+- Content-Type: `application/json` (JSON)
+- Encoding: `UTF8`
+
+With body (example values):
+##### Request Body
+```json
+{
+    "mxid": "@john.doe:example.org",
+    "localpart": "john.doe",
+    "domain": "example.org"
+}
+```
+#### Response Body
+For all responses, the same object structure will be parsed, making the non-relevant fields as optional.
+
+Structure with example values:
+```json
+{
+  "profile": {
+    "display_name": "John Doe",
+    "threepids": [
+      {
+        "medium": "email",
+        "address": "john.doe@example.org"
+      },
+      {
+        "...": "..."
+      }
+    ],
+    "roles": [
+      "DomainUsers",
+      "SalesOrg",
+      "..."
+    ]
+  }
+}
+```
+The base `profile` key is mandatory. `display_name`, `threepids` and `roles` are only to be returned on the relevant request.

src/main/java/io/kamax/mxisd/as/AppServiceHandler.java

@@ -28,18 +28,24 @@ import io.kamax.matrix._ThreePid;
 import io.kamax.matrix.event.EventKey;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.backend.sql.synapse.Synapse;
+import io.kamax.mxisd.config.ListenerConfig;
 import io.kamax.mxisd.config.MatrixConfig;
 import io.kamax.mxisd.notification.NotificationManager;
 import io.kamax.mxisd.profile.ProfileManager;
+import io.kamax.mxisd.storage.IStorage;
+import io.kamax.mxisd.storage.ormlite.dao.ASTransactionDao;
+import io.kamax.mxisd.util.GsonParser;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Component;
 
-import java.util.HashMap;
-import java.util.List;
-import java.util.Map;
+import java.io.InputStream;
+import java.time.Instant;
+import java.util.*;
+import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
 import java.util.stream.Collectors;
 
 @Component
@@ -47,20 +53,83 @@ public class AppServiceHandler {
 
     private final Logger log = LoggerFactory.getLogger(AppServiceHandler.class);
 
+    private final GsonParser parser;
+
+    private String localpart;
     private MatrixConfig cfg;
+    private IStorage store;
     private ProfileManager profiler;
     private NotificationManager notif;
     private Synapse synapse;
 
+    private Map<String, CompletableFuture<String>> transactionsInProgress;
+
     @Autowired
-    public AppServiceHandler(MatrixConfig cfg, ProfileManager profiler, NotificationManager notif, Synapse synapse) {
+    public AppServiceHandler(ListenerConfig lCfg, MatrixConfig cfg, IStorage store, ProfileManager profiler, NotificationManager notif, Synapse synapse) {
         this.cfg = cfg;
+        this.store = store;
         this.profiler = profiler;
         this.notif = notif;
         this.synapse = synapse;
+
+        localpart = lCfg.getLocalpart();
+        parser = new GsonParser();
+        transactionsInProgress = new ConcurrentHashMap<>();
+    }
+
+    public CompletableFuture<String> processTransaction(String txnId, InputStream is) {
+        synchronized (this) {
+            Optional<ASTransactionDao> dao = store.getTransactionResult(localpart, txnId);
+            if (dao.isPresent()) {
+                log.info("AS Transaction {} already processed - returning computed result", txnId);
+                return CompletableFuture.completedFuture(dao.get().getResult());
+            }
+
+            CompletableFuture<String> f = transactionsInProgress.get(txnId);
+            if (Objects.nonNull(f)) {
+                log.info("Returning future for transaction {}", txnId);
+                return f;
+            }
+
+            transactionsInProgress.put(txnId, new CompletableFuture<>());
+        }
+
+        CompletableFuture<String> future = transactionsInProgress.get(txnId);
+
+        Instant start = Instant.now();
+        log.info("Processing AS Transaction {}: start", txnId);
+        try {
+            List<JsonObject> events = GsonUtil.asList(GsonUtil.getArray(parser.parse(is), "events"), JsonObject.class);
+            is.close();
+            log.debug("{} event(s) parsed", events.size());
+
+            processTransaction(events);
+            Instant end = Instant.now();
+            log.info("Processed AS transaction {} in {} ms", txnId, (Instant.now().toEpochMilli() - start.toEpochMilli()));
+
+            String result = "{}";
+
+            try {
+                log.info("Saving transaction details to store");
+                store.insertTransactionResult(localpart, txnId, end, result);
+            } finally {
+                log.debug("Removing CompletedFuture from transaction map");
+                transactionsInProgress.remove(txnId);
+            }
+
+            future.complete(result);
+        } catch (Exception e) {
+            log.error("Unable to properly process transaction {}", txnId, e);
+            future.completeExceptionally(e);
+        }
+
+        log.info("Processing AS Transaction {}: end", txnId);
+        return future;
     }
 
     public void processTransaction(List<JsonObject> eventsJson) {
+        log.info("Processing transaction events: start");
+
         eventsJson.forEach(ev -> {
             String evId = EventKey.Id.getStringOrNull(ev);
             if (StringUtils.isBlank(evId)) {
@@ -78,10 +147,11 @@ public class AppServiceHandler {
 
             String senderId = EventKey.Sender.getStringOrNull(ev);
             if (StringUtils.isBlank(senderId)) {
-                log.debug("Event has no room ID, skipping");
+                log.debug("Event has no sender ID, skipping");
                 return;
             }
             _MatrixID sender = MatrixID.asAcceptable(senderId);
+            log.debug("Sender: {}", senderId);
 
             if (!StringUtils.equals("m.room.member", GsonUtil.getStringOrNull(ev, "type"))) {
                 log.debug("This is not a room membership event, skipping");
@@ -105,7 +175,7 @@ public class AppServiceHandler {
                 return;
             }
 
-            log.info("Got invite for {}", inviteeId);
+            log.info("Got invite from {} to {}", senderId, inviteeId);
 
             boolean wasSent = false;
             List<_ThreePid> tpids = profiler.getThreepids(invitee).stream()
@@ -121,7 +191,7 @@ public class AppServiceHandler {
                     synapse.getRoomName(roomId).ifPresent(name -> properties.put("room_name", name));
                 } catch (RuntimeException e) {
                     log.warn("Could not fetch room name", e);
-                    log.warn("Unable to fetch room name: Did you integrate your Homeserver as documented?");
+                    log.info("Unable to fetch room name: Did you integrate your Homeserver as documented?");
                 }
 
                 IMatrixIdInvite inv = new MatrixIdInvite(roomId, sender, invitee, tpid.getMedium(), tpid.getAddress(), properties);
@@ -134,6 +204,8 @@ public class AppServiceHandler {
 
             log.debug("Event {}: processing end", evId);
         });
+
+        log.info("Processing transaction events: end");
     }
 
 }

src/main/java/io/kamax/mxisd/auth/AuthManager.java

@@ -20,37 +20,96 @@
 
 package io.kamax.mxisd.auth;
 
+import com.google.gson.Gson;
+import com.google.gson.JsonObject;
+import com.google.gson.JsonParser;
+import com.google.gson.JsonSyntaxException;
+import com.google.i18n.phonenumbers.NumberParseException;
+import com.google.i18n.phonenumbers.PhoneNumberUtil;
+import com.google.i18n.phonenumbers.Phonenumber;
 import io.kamax.matrix.MatrixID;
 import io.kamax.matrix.ThreePid;
 import io.kamax.matrix._MatrixID;
 import io.kamax.matrix._ThreePid;
+import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.UserIdType;
 import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import io.kamax.mxisd.config.AuthenticationConfig;
 import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.dns.ClientDnsOverwrite;
+import io.kamax.mxisd.exception.RemoteLoginException;
 import io.kamax.mxisd.invitation.InvitationManager;
 import io.kamax.mxisd.lookup.ThreePidMapping;
+import io.kamax.mxisd.lookup.strategy.LookupStrategy;
+import io.kamax.mxisd.util.RestClientUtils;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.HttpEntity;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.util.EntityUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.stereotype.Service;
 
+import java.io.IOException;
+import java.net.URI;
+import java.nio.charset.StandardCharsets;
 import java.util.ArrayList;
 import java.util.List;
+import java.util.Objects;
 
 @Service
 public class AuthManager {
 
-    private Logger log = LoggerFactory.getLogger(AuthManager.class);
+    private static final String TypeKey = "type";
+    private static final String UserKey = "user";
+    private static final String IdentifierKey = "identifier";
+    private static final String ThreepidMediumKey = "medium";
+    private static final String ThreepidAddressKey = "address";
+    private static final String UserIdTypeValue = "m.id.user";
+    private static final String ThreepidTypeValue = "m.id.thirdparty";
 
-    @Autowired
-    private List<AuthenticatorProvider> providers = new ArrayList<>();
+    private final Logger log = LoggerFactory.getLogger(AuthManager.class);
+    private final Gson gson = GsonUtil.get();
 
-    @Autowired
+    private List<AuthenticatorProvider> providers;
     private MatrixConfig mxCfg;
+    private AuthenticationConfig cfg;
+    private InvitationManager invMgr;
+    private ClientDnsOverwrite dns;
+    private LookupStrategy strategy;
+    private CloseableHttpClient client;
 
     @Autowired
-    private InvitationManager invMgr;
+    public AuthManager(
+            AuthenticationConfig cfg,
+            MatrixConfig mxCfg,
+            List<AuthenticatorProvider> providers,
+            LookupStrategy strategy,
+            InvitationManager invMgr,
+            ClientDnsOverwrite dns,
+            CloseableHttpClient client
+    ) {
+        this.cfg = cfg;
+        this.mxCfg = mxCfg;
+        this.providers = new ArrayList<>(providers);
+        this.strategy = strategy;
+        this.invMgr = invMgr;
+        this.dns = dns;
+        this.client = client;
+    }
+
+    public String resolveProxyUrl(URI target) {
+        URIBuilder builder = dns.transform(target);
+        String urlToLogin = builder.toString();
+        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
+        return urlToLogin;
+    }
 
     public UserAuthResult authenticate(String id, String password) {
         _MatrixID mxid = MatrixID.asAcceptable(id);
@@ -92,4 +151,128 @@ public class AuthManager {
         return new UserAuthResult().failure();
     }
 
+    public String proxyLogin(URI target, String body) {
+        JsonObject reqJsonObject = io.kamax.matrix.json.GsonUtil.parseObj(body);
+
+        GsonUtil.findObj(reqJsonObject, IdentifierKey).ifPresent(obj -> {
+            GsonUtil.findString(obj, TypeKey).ifPresent(type -> {
+                if (StringUtils.equals(type, UserIdTypeValue)) {
+                    log.info("Login request is User ID type");
+
+                    if (cfg.getRewrite().getUser().getRules().isEmpty()) {
+                        log.info("No User ID rewrite rules to apply");
+                    } else {
+                        log.info("User ID rewrite rules: checking for a match");
+
+                        String userId = GsonUtil.getStringOrThrow(obj, UserKey);
+                        for (AuthenticationConfig.Rule m : cfg.getRewrite().getUser().getRules()) {
+                            if (m.getPattern().matcher(userId).matches()) {
+                                log.info("Found matching pattern, resolving to 3PID with medium {}", m.getMedium());
+
+                                // Remove deprecated login info on the top object if exists to avoid duplication
+                                reqJsonObject.remove(UserKey);
+                                obj.addProperty(TypeKey, ThreepidTypeValue);
+                                obj.addProperty(ThreepidMediumKey, m.getMedium());
+                                obj.addProperty(ThreepidAddressKey, userId);
+
+                                log.info("Rewrite to 3PID done");
+                            }
+                        }
+
+                        log.info("User ID rewrite rules: done checking rules");
+                    }
+                }
+            });
+        });
+
+        GsonUtil.findObj(reqJsonObject, IdentifierKey).ifPresent(obj -> {
+            GsonUtil.findString(obj, TypeKey).ifPresent(type -> {
+                if (StringUtils.equals(type, ThreepidTypeValue)) {
+                    // Remove deprecated login info if exists to avoid duplication
+                    reqJsonObject.remove(ThreepidMediumKey);
+                    reqJsonObject.remove(ThreepidAddressKey);
+
+                    GsonUtil.findPrimitive(obj, ThreepidMediumKey).ifPresent(medium -> {
+                        GsonUtil.findPrimitive(obj, ThreepidAddressKey).ifPresent(address -> {
+                            log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
+                            strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
+                                obj.remove(ThreepidMediumKey);
+                                obj.remove(ThreepidAddressKey);
+                                obj.addProperty(TypeKey, UserIdTypeValue);
+                                obj.addProperty(UserKey, lookupDataOpt.getMxid().getLocalPart());
+                            });
+                        });
+                    });
+                }
+
+                if (StringUtils.equals(type, "m.id.phone")) {
+                    // Remove deprecated login info if exists to avoid duplication
+                    reqJsonObject.remove(ThreepidMediumKey);
+                    reqJsonObject.remove(ThreepidAddressKey);
+
+                    GsonUtil.findPrimitive(obj, "number").ifPresent(number -> {
+                        GsonUtil.findPrimitive(obj, "country").ifPresent(country -> {
+                            log.info("Login request with phone '{}'-'{}'", country.getAsString(), number.getAsString());
+                            try {
+                                PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
+                                Phonenumber.PhoneNumber phoneNumber = phoneUtil.parse(number.getAsString(), country.getAsString());
+                                String msisdn = phoneUtil.format(phoneNumber, PhoneNumberUtil.PhoneNumberFormat.E164).replace("+", "");
+                                String medium = "msisdn";
+                                strategy.findLocal(medium, msisdn).ifPresent(lookupDataOpt -> {
+                                    obj.remove("country");
+                                    obj.remove("number");
+                                    obj.addProperty(TypeKey, UserIdTypeValue);
+                                    obj.addProperty(UserKey, lookupDataOpt.getMxid().getLocalPart());
+                                });
+                            } catch (NumberParseException e) {
+                                log.error("Not a valid phone number");
+                                throw new RuntimeException(e);
+                            }
+                        });
+                    });
+                }
+            });
+        });
+
+        // invoke 'login' on homeserver
+        HttpPost httpPost = RestClientUtils.post(resolveProxyUrl(target), gson, reqJsonObject);
+        try (CloseableHttpResponse httpResponse = client.execute(httpPost)) {
+            // check http status
+            int status = httpResponse.getStatusLine().getStatusCode();
+            log.info("http status = {}", status);
+            if (status != 200) {
+                // try to get possible json error message from response
+                // otherwise just get returned plain error message
+                String errcode = String.valueOf(httpResponse.getStatusLine().getStatusCode());
+                String error = EntityUtils.toString(httpResponse.getEntity());
+                if (httpResponse.getEntity() != null) {
+                    try {
+                        JsonObject bodyJson = new JsonParser().parse(error).getAsJsonObject();
+                        if (bodyJson.has("errcode")) {
+                            errcode = bodyJson.get("errcode").getAsString();
+                        }
+                        if (bodyJson.has("error")) {
+                            error = bodyJson.get("error").getAsString();
+                        }
+                        throw new RemoteLoginException(status, errcode, error, bodyJson);
+                    } catch (JsonSyntaxException e) {
+                        log.warn("Response body is not JSON");
+                    }
+                }
+                throw new RemoteLoginException(status, errcode, error);
+            }
+
+            // return response
+            HttpEntity entity = httpResponse.getEntity();
+            if (Objects.isNull(entity)) {
+                log.warn("Expected HS to return data but got nothing");
+                return "";
+            } else {
+                return IOUtils.toString(httpResponse.getEntity().getContent(), StandardCharsets.UTF_8);
+            }
+        } catch (IOException e) {
+            throw new RuntimeException(e);
+        }
+    }
+
 }

src/main/java/io/kamax/mxisd/backend/ldap/LdapAuthProvider.java

@@ -30,6 +30,7 @@ import io.kamax.mxisd.auth.provider.AuthenticatorProvider;
 import io.kamax.mxisd.auth.provider.BackendAuthResult;
 import io.kamax.mxisd.config.MatrixConfig;
 import io.kamax.mxisd.config.ldap.LdapConfig;
+import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.util.GsonUtil;
 import org.apache.commons.lang.StringUtils;
 import org.apache.directory.api.ldap.model.cursor.CursorException;
@@ -87,7 +88,6 @@ public class LdapAuthProvider extends LdapBackend implements AuthenticatorProvid
     public BackendAuthResult authenticate(_MatrixID mxid, String password) {
         log.info("Performing auth for {}", mxid);
 
-
         try (LdapConnection conn = getConn()) {
             bind(conn);
 
@@ -108,62 +108,65 @@ public class LdapAuthProvider extends LdapBackend implements AuthenticatorProvid
             String[] attArray = new String[attributes.size()];
             attributes.toArray(attArray);
 
-            log.debug("Base DN: {}", getBaseDn());
             log.debug("Query: {}", userFilter);
             log.debug("Attributes: {}", GsonUtil.build().toJson(attArray));
 
-            try (EntryCursor cursor = conn.search(getBaseDn(), userFilter, SearchScope.SUBTREE, attArray)) {
-                while (cursor.next()) {
-                    Entry entry = cursor.get();
-                    String dn = entry.getDn().getName();
-                    log.info("Checking possible match, DN: {}", dn);
+            for (String baseDN : getBaseDNs()) {
+                log.debug("Base DN: {}", baseDN);
 
-                    if (!getAttribute(entry, getUidAtt()).isPresent()) {
-                        continue;
-                    }
+                try (EntryCursor cursor = conn.search(baseDN, userFilter, SearchScope.SUBTREE, attArray)) {
+                    while (cursor.next()) {
+                        Entry entry = cursor.get();
+                        String dn = entry.getDn().getName();
+                        log.info("Checking possible match, DN: {}", dn);
 
-                    log.info("Attempting authentication on LDAP for {}", dn);
-                    try {
-                        conn.bind(entry.getDn(), password);
-                    } catch (LdapException e) {
-                        log.info("Unable to bind using {} because {}", entry.getDn().getName(), e.getMessage());
-                        return BackendAuthResult.failure();
-                    }
+                        if (!getAttribute(entry, getUidAtt()).isPresent()) {
+                            continue;
+                        }
 
-                    Attribute nameAttribute = entry.get(getAt().getName());
-                    String name = nameAttribute != null ? nameAttribute.get().toString() : null;
+                        log.info("Attempting authentication on LDAP for {}", dn);
+                        try {
+                            conn.bind(entry.getDn(), password);
+                        } catch (LdapException e) {
+                            log.info("Unable to bind using {} because {}", entry.getDn().getName(), e.getMessage());
+                            return BackendAuthResult.failure();
+                        }
 
-                    log.info("Authentication successful for {}", entry.getDn().getName());
-                    log.info("DN {} is a valid match", dn);
+                        Attribute nameAttribute = entry.get(getAt().getName());
+                        String name = nameAttribute != null ? nameAttribute.get().toString() : null;
 
-                    // TODO should we canonicalize the MXID?
-                    BackendAuthResult result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
-                    log.info("Processing 3PIDs for profile");
-                    getAt().getThreepid().forEach((k, v) -> {
-                        log.info("Processing 3PID type {}", k);
-                        v.forEach(attId -> {
-                            List<String> values = getAttributes(entry, attId);
-                            log.info("\tAttribute {} has {} value(s)", attId, values.size());
-                            getAttributes(entry, attId).forEach(tpidValue -> {
-                                if (ThreePidMedium.PhoneNumber.is(k)) {
-                                    tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
-                                }
-                                result.withThreePid(new ThreePid(k, tpidValue));
+                        log.info("Authentication successful for {}", entry.getDn().getName());
+                        log.info("DN {} is a valid match", dn);
+
+                        // TODO should we canonicalize the MXID?
+                        BackendAuthResult result = BackendAuthResult.success(mxid.getId(), UserIdType.MatrixID, name);
+                        log.info("Processing 3PIDs for profile");
+                        getAt().getThreepid().forEach((k, v) -> {
+                            log.info("Processing 3PID type {}", k);
+                            v.forEach(attId -> {
+                                List<String> values = getAttributes(entry, attId);
+                                log.info("\tAttribute {} has {} value(s)", attId, values.size());
+                                getAttributes(entry, attId).forEach(tpidValue -> {
+                                    if (ThreePidMedium.PhoneNumber.is(k)) {
+                                        tpidValue = getMsisdn(tpidValue).orElse(tpidValue);
+                                    }
+                                    result.withThreePid(new ThreePid(k, tpidValue));
+                                });
                             });
                         });
-                    });
 
-                    log.info("Found {} 3PIDs", result.getProfile().getThreePids().size());
-                    return result;
+                        log.info("Found {} 3PIDs", result.getProfile().getThreePids().size());
+                        return result;
+                    }
+                } catch (CursorLdapReferralException e) {
+                    log.warn("Entity for {} is only available via referral, skipping", mxid);
                 }
-            } catch (CursorLdapReferralException e) {
-                log.warn("Entity for {} is only available via referral, skipping", mxid);
             }
 
             log.info("No match were found for {}", mxid);
             return BackendAuthResult.failure();
         } catch (LdapException | IOException | CursorException e) {
-            throw new RuntimeException(e);
+            throw new InternalServerError(e);
         }
     }
 

src/main/java/io/kamax/mxisd/backend/ldap/LdapBackend.java

@@ -59,8 +59,8 @@ public abstract class LdapBackend {
         return cfg;
     }
 
-    protected String getBaseDn() {
-        return cfg.getConnection().getBaseDn();
+    protected List<String> getBaseDNs() {
+        return cfg.getConnection().getBaseDNs();
     }
 
     protected LdapConfig.Attribute getAt() {

src/main/java/io/kamax/mxisd/backend/ldap/LdapDirectoryProvider.java

@@ -65,34 +65,37 @@ public class LdapDirectoryProvider extends LdapBackend implements IDirectoryProv
             bind(conn);
 
             LdapConfig.Attribute atCfg = getCfg().getAttribute();
-
             attributes = new ArrayList<>(attributes);
             attributes.add(getUidAtt());
             String[] attArray = new String[attributes.size()];
             attributes.toArray(attArray);
             String searchQuery = buildOrQueryWithFilter(getCfg().getDirectory().getFilter(), "*" + query + "*", attArray);
 
-            log.debug("Base DN: {}", getBaseDn());
             log.debug("Query: {}", searchQuery);
             log.debug("Attributes: {}", GsonUtil.build().toJson(attArray));
 
-            try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, attArray)) {
-                while (cursor.next()) {
-                    Entry entry = cursor.get();
-                    log.info("Found possible match, DN: {}", entry.getDn().getName());
-                    getAttribute(entry, getUidAtt()).ifPresent(uid -> {
-                        log.info("DN {} is a valid match", entry.getDn().getName());
-                        try {
-                            UserDirectorySearchResult.Result entryResult = new UserDirectorySearchResult.Result();
-                            entryResult.setUserId(buildMatrixIdFromUid(uid));
-                            getAttribute(entry, atCfg.getName()).ifPresent(entryResult::setDisplayName);
-                            result.addResult(entryResult);
-                        } catch (IllegalArgumentException e) {
-                            log.warn("Bind was found but type {} is not supported", atCfg.getUid().getType());
-                        }
-                    });
+            for (String baseDN : getBaseDNs()) {
+                log.debug("Base DN: {}", baseDN);
+
+                try (EntryCursor cursor = conn.search(baseDN, searchQuery, SearchScope.SUBTREE, attArray)) {
+                    while (cursor.next()) {
+                        Entry entry = cursor.get();
+                        log.info("Found possible match, DN: {}", entry.getDn().getName());
+                        getAttribute(entry, getUidAtt()).ifPresent(uid -> {
+                            log.info("DN {} is a valid match", entry.getDn().getName());
+                            try {
+                                UserDirectorySearchResult.Result entryResult = new UserDirectorySearchResult.Result();
+                                entryResult.setUserId(buildMatrixIdFromUid(uid));
+                                getAttribute(entry, atCfg.getName()).ifPresent(entryResult::setDisplayName);
+                                result.addResult(entryResult);
+                            } catch (IllegalArgumentException e) {
+                                log.warn("Bind was found but type {} is not supported", atCfg.getUid().getType());
+                            }
+                        });
+                    }
                 }
             }
+
         } catch (CursorLdapReferralException e) {
             log.warn("An entry is only available via referral, skipping");
         } catch (IOException | LdapException | CursorException e) {

src/main/java/io/kamax/mxisd/backend/ldap/LdapProfileProvider.java

@@ -69,32 +69,33 @@ public class LdapProfileProvider extends LdapBackend implements ProfileProvider
             bind(conn);
 
             String searchQuery = buildOrQueryWithFilter(getCfg().getProfile().getFilter(), uid, getUidAtt());
-
-            log.debug("Base DN: {}", getBaseDn());
             log.debug("Query: {}", searchQuery);
 
-            try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, getAt().getName())) {
-                while (cursor.next()) {
-                    Entry entry = cursor.get();
-                    log.info("Found possible match, DN: {}", entry.getDn().getName());
-                    Optional<String> v = getAttribute(entry, getAt().getName()).flatMap(id -> {
-                        log.info("DN {} is a valid match", entry.getDn().getName());
-                        try {
-                            return getAttribute(entry, getAt().getName());
-                        } catch (IllegalArgumentException e) {
-                            log.warn("Bind was found but type {} is not supported", getAt().getUid().getType());
-                            return Optional.empty();
-                        }
-                    });
+            for (String baseDN : getBaseDNs()) {
+                log.debug("Base DN: {}", baseDN);
+                try (EntryCursor cursor = conn.search(baseDN, searchQuery, SearchScope.SUBTREE, getAt().getName())) {
+                    while (cursor.next()) {
+                        Entry entry = cursor.get();
+                        log.info("Found possible match, DN: {}", entry.getDn().getName());
+                        Optional<String> v = getAttribute(entry, getAt().getName()).flatMap(id -> {
+                            log.info("DN {} is a valid match", entry.getDn().getName());
+                            try {
+                                return getAttribute(entry, getAt().getName());
+                            } catch (IllegalArgumentException e) {
+                                log.warn("Bind was found but type {} is not supported", getAt().getUid().getType());
+                                return Optional.empty();
+                            }
+                        });
 
-                    if (v.isPresent()) {
-                        log.info("DN {} is the final match", entry.getDn().getName());
-                        return v;
+                        if (v.isPresent()) {
+                            log.info("DN {} is the final match", entry.getDn().getName());
+                            return v;
+                        }
                     }
+                } catch (CursorLdapReferralException e) {
+                    log.warn("An entry is only available via referral, skipping");
                 }
             }
-        } catch (CursorLdapReferralException e) {
-            log.warn("An entry is only available via referral, skipping");
         } catch (IOException | LdapException | CursorException e) {
             throw new InternalServerError(e);
         }
@@ -111,7 +112,6 @@ public class LdapProfileProvider extends LdapBackend implements ProfileProvider
         try (LdapConnection conn = getConn()) {
             bind(conn);
 
-            log.debug("Base DN: {}", getBaseDn());
             getCfg().getAttribute().getThreepid().forEach((medium, attributes) -> {
                 String[] attArray = new String[attributes.size()];
                 attributes.toArray(attArray);
@@ -120,28 +120,30 @@ public class LdapProfileProvider extends LdapBackend implements ProfileProvider
 
                 log.debug("Query for 3PID {}: {}", medium, searchQuery);
 
-                try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, attArray)) {
-                    while (cursor.next()) {
-                        Entry entry = cursor.get();
-                        log.info("Found possible match, DN: {}", entry.getDn().getName());
-                        try {
-                            attributes.stream()
-                                    .flatMap(at -> getAttributes(entry, at).stream())
-                                    .forEach(address -> {
-                                        log.info("Found 3PID: {} - {}", medium, address);
-                                        threePids.add(new ThreePid(medium, address));
-                                    });
-                        } catch (IllegalArgumentException e) {
-                            log.warn("Bind was found but type {} is not supported", getAt().getUid().getType());
+                for (String baseDN : getBaseDNs()) {
+                    log.debug("Base DN: {}", baseDN);
+                    try (EntryCursor cursor = conn.search(baseDN, searchQuery, SearchScope.SUBTREE, attArray)) {
+                        while (cursor.next()) {
+                            Entry entry = cursor.get();
+                            log.info("Found possible match, DN: {}", entry.getDn().getName());
+                            try {
+                                attributes.stream()
+                                        .flatMap(at -> getAttributes(entry, at).stream())
+                                        .forEach(address -> {
+                                            log.info("Found 3PID: {} - {}", medium, address);
+                                            threePids.add(new ThreePid(medium, address));
+                                        });
+                            } catch (IllegalArgumentException e) {
+                                log.warn("Bind was found but type {} is not supported", getAt().getUid().getType());
+                            }
                         }
+                    } catch (CursorLdapReferralException e) {
+                        log.warn("An entry is only available via referral, skipping");
+                    } catch (LdapException | IOException | CursorException e) {
+                        throw new InternalServerError(e);
                     }
-                } catch (CursorLdapReferralException e) {
-                    log.warn("An entry is only available via referral, skipping");
-                } catch (IOException | LdapException | CursorException e) {
-                    throw new InternalServerError(e);
                 }
             });
-
         } catch (IOException | LdapException e) {
             throw new InternalServerError(e);
         }

src/main/java/io/kamax/mxisd/backend/ldap/LdapThreePidProvider.java

@@ -78,28 +78,30 @@ public class LdapThreePidProvider extends LdapBackend implements IThreePidProvid
         // we merge 3PID specific query with global/specific filter, if one exists.
         String tPidQuery = tPidQueryOpt.get().replaceAll(getCfg().getIdentity().getToken(), value);
         String searchQuery = buildWithFilter(tPidQuery, getCfg().getIdentity().getFilter());
-
-        log.debug("Base DN: {}", getBaseDn());
         log.debug("Query: {}", searchQuery);
         log.debug("Attributes: {}", GsonUtil.build().toJson(getUidAtt()));
 
-        try (EntryCursor cursor = conn.search(getBaseDn(), searchQuery, SearchScope.SUBTREE, getUidAtt())) {
-            while (cursor.next()) {
-                Entry entry = cursor.get();
-                log.info("Found possible match, DN: {}", entry.getDn().getName());
+        for (String baseDN : getBaseDNs()) {
+            log.debug("Base DN: {}", baseDN);
 
-                Optional<String> data = getAttribute(entry, getUidAtt());
-                if (!data.isPresent()) {
-                    continue;
+            try (EntryCursor cursor = conn.search(baseDN, searchQuery, SearchScope.SUBTREE, getUidAtt())) {
+                while (cursor.next()) {
+                    Entry entry = cursor.get();
+                    log.info("Found possible match, DN: {}", entry.getDn().getName());
+
+                    Optional<String> data = getAttribute(entry, getUidAtt());
+                    if (!data.isPresent()) {
+                        continue;
+                    }
+
+                    log.info("DN {} is a valid match", entry.getDn().getName());
+                    return Optional.of(buildMatrixIdFromUid(data.get()));
                 }
-
-                log.info("DN {} is a valid match", entry.getDn().getName());
-                return Optional.of(buildMatrixIdFromUid(data.get()));
+            } catch (CursorLdapReferralException e) {
+                log.warn("3PID {} is only available via referral, skipping", value);
+            } catch (IOException | LdapException | CursorException e) {
+                throw new InternalServerError(e);
             }
-        } catch (CursorLdapReferralException e) {
-            log.warn("3PID {} is only available via referral, skipping", value);
-        } catch (IOException | LdapException | CursorException e) {
-            throw new InternalServerError(e);
         }
 
         return Optional.empty();

src/main/java/io/kamax/mxisd/backend/rest/RestProfileProvider.java

@@ -0,0 +1,147 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.rest;
+
+import com.google.gson.JsonObject;
+import com.google.gson.JsonSyntaxException;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix._ThreePid;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.matrix.json.InvalidJsonException;
+import io.kamax.mxisd.config.rest.RestBackendConfig;
+import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.profile.JsonProfileRequest;
+import io.kamax.mxisd.profile.JsonProfileResult;
+import io.kamax.mxisd.profile.ProfileProvider;
+import org.apache.commons.io.IOUtils;
+import org.apache.commons.lang.StringUtils;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.springframework.stereotype.Component;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.nio.charset.StandardCharsets;
+import java.util.*;
+import java.util.function.Function;
+
+@Component
+public class RestProfileProvider extends RestProvider implements ProfileProvider {
+
+    private final Logger log = LoggerFactory.getLogger(RestProfileProvider.class);
+
+    public RestProfileProvider(RestBackendConfig cfg) {
+        super(cfg);
+    }
+
+    @Override
+    public boolean isEnabled() {
+        return cfg.isEnabled() && cfg.getEndpoints().getProfile().isPresent();
+    }
+
+    private <T> Optional<T> doRequest(
+            _MatrixID userId,
+            Function<RestBackendConfig.ProfileEndpoints, Optional<String>> endpoint,
+            Function<JsonProfileResult, Optional<T>> value
+    ) {
+        return cfg.getEndpoints().getProfile()
+                // We get the endpoint
+                .flatMap(endpoint)
+                // We only continue if there is a value
+                .filter(StringUtils::isNotBlank)
+                // We use the endpoint
+                .flatMap(url -> {
+                    try {
+                        URIBuilder builder = new URIBuilder(url);
+                        HttpPost req = new HttpPost(builder.build());
+                        req.setEntity(new StringEntity(GsonUtil.get().toJson(new JsonProfileRequest(userId)), ContentType.APPLICATION_JSON));
+                        try (CloseableHttpResponse res = client.execute(req)) {
+                            int sc = res.getStatusLine().getStatusCode();
+                            if (sc == 404) {
+                                log.info("Got 404 - No result found");
+                                return Optional.empty();
+                            }
+
+                            if (sc != 200) {
+                                throw new InternalServerError("Unexpected backed status code: " + sc);
+                            }
+
+                            String body = IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8);
+                            if (StringUtils.isBlank(body)) {
+                                log.warn("Backend response body is empty/blank, expected JSON object with profile key");
+                                return Optional.empty();
+                            }
+
+                            Optional<JsonObject> pJson = GsonUtil.findObj(GsonUtil.parseObj(body), "profile");
+                            if (!pJson.isPresent()) {
+                                log.warn("Backend response body is invalid, expected JSON object with profile key");
+                                return Optional.empty();
+                            }
+
+                            JsonProfileResult profile = gson.fromJson(pJson.get(), JsonProfileResult.class);
+                            return value.apply(profile);
+                        }
+                    } catch (JsonSyntaxException | InvalidJsonException e) {
+                        log.error("Unable to parse backend response as JSON", e);
+                        throw new InternalServerError(e);
+                    } catch (URISyntaxException e) {
+                        log.error("Unable to build a valid request URL", e);
+                        throw new InternalServerError(e);
+                    } catch (IOException e) {
+                        log.error("I/O Error during backend request", e);
+                        throw new InternalServerError();
+                    }
+                });
+    }
+
+    @Override
+    public Optional<String> getDisplayName(_MatrixID userId) {
+        return doRequest(userId, p -> Optional.ofNullable(p.getDisplayName()), profile -> Optional.ofNullable(profile.getDisplayName()));
+    }
+
+    @Override
+    public List<_ThreePid> getThreepids(_MatrixID userId) {
+        return doRequest(userId, p -> Optional.ofNullable(p.getThreepids()), profile -> {
+            List<_ThreePid> t = new ArrayList<>();
+            if (Objects.nonNull(profile.getThreepids())) {
+                t.addAll(profile.getThreepids());
+            }
+            return Optional.of(t);
+        }).orElseGet(Collections::emptyList);
+    }
+
+    @Override
+    public List<String> getRoles(_MatrixID userId) {
+        return doRequest(userId, p -> Optional.ofNullable(p.getRoles()), profile -> {
+            List<String> t = new ArrayList<>();
+            if (Objects.nonNull(profile.getRoles())) {
+                t.addAll(profile.getRoles());
+            }
+            return Optional.of(t);
+        }).orElseGet(Collections::emptyList);
+    }
+
+}

src/main/java/io/kamax/mxisd/config/AsyncConfig.java

@@ -0,0 +1,36 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import org.springframework.context.annotation.Configuration;
+import org.springframework.web.servlet.config.annotation.AsyncSupportConfigurer;
+import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
+
+@Configuration
+public class AsyncConfig extends WebMvcConfigurerAdapter {
+
+    @Override
+    public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
+        configurer.setDefaultTimeout(60 * 60 * 1000); // 1h in milliseconds
+        super.configureAsyncSupport(configurer);
+    }
+
+}

src/main/java/io/kamax/mxisd/config/AuthenticationConfig.java

@@ -0,0 +1,110 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.config;
+
+import org.springframework.boot.context.properties.ConfigurationProperties;
+import org.springframework.context.annotation.Configuration;
+
+import javax.annotation.PostConstruct;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.regex.Pattern;
+
+@Configuration
+@ConfigurationProperties(prefix = "auth")
+public class AuthenticationConfig {
+
+    public static class Rule {
+
+        private String regex;
+        private transient Pattern pattern;
+        private String medium;
+
+        public String getRegex() {
+            return regex;
+        }
+
+        public void setRegex(String regex) {
+            this.regex = regex;
+        }
+
+        public Pattern getPattern() {
+            return pattern;
+        }
+
+        public void setPattern(Pattern pattern) {
+            this.pattern = pattern;
+        }
+
+        public String getMedium() {
+            return medium;
+        }
+
+        public void setMedium(String medium) {
+            this.medium = medium;
+        }
+
+    }
+
+    public static class User {
+
+        private List<Rule> rules = new ArrayList<>();
+
+        public List<Rule> getRules() {
+            return rules;
+        }
+
+        public void setRules(List<Rule> mappings) {
+            this.rules = mappings;
+        }
+
+    }
+
+    public static class Rewrite {
+
+        private User user = new User();
+
+        public User getUser() {
+            return user;
+        }
+
+        public void setUser(User user) {
+            this.user = user;
+        }
+
+    }
+
+    private Rewrite rewrite = new Rewrite();
+
+    public Rewrite getRewrite() {
+        return rewrite;
+    }
+
+    public void setRewrite(Rewrite rewrite) {
+        this.rewrite = rewrite;
+    }
+
+    @PostConstruct
+    public void build() {
+        getRewrite().getUser().getRules().forEach(mapping -> mapping.setPattern(Pattern.compile(mapping.getRegex())));
+    }
+
+}

src/main/java/io/kamax/mxisd/config/ldap/LdapConfig.java

@@ -110,6 +110,7 @@ public abstract class LdapConfig {
         private String bindDn;
         private String bindPassword;
         private String baseDn;
+        private List<String> baseDNs = new ArrayList<>();
 
         public boolean isTls() {
             return tls;
@@ -151,14 +152,24 @@ public abstract class LdapConfig {
             this.bindPassword = bindPassword;
         }
 
+        @Deprecated
         public String getBaseDn() {
             return baseDn;
         }
 
+        @Deprecated
         public void setBaseDn(String baseDn) {
             this.baseDn = baseDn;
         }
 
+        public List<String> getBaseDNs() {
+            return baseDNs;
+        }
+
+        public void setBaseDNs(List<String> baseDNs) {
+            this.baseDNs = baseDNs;
+        }
+
     }
 
     public static class Directory {
@@ -253,11 +264,11 @@ public abstract class LdapConfig {
     private boolean enabled;
     private String filter;
 
-    private Connection connection;
-    private Attribute attribute;
-    private Auth auth;
-    private Directory directory;
-    private Identity identity;
+    private Connection connection = new Connection();
+    private Attribute attribute = new Attribute();
+    private Auth auth = new Auth();
+    private Directory directory = new Directory();
+    private Identity identity = new Identity();
     private Profile profile = new Profile();
 
     protected abstract String getConfigName();
@@ -343,8 +354,14 @@ public abstract class LdapConfig {
             throw new IllegalStateException("LDAP port is not valid");
         }
 
-        if (StringUtils.isBlank(connection.getBaseDn())) {
-            throw new ConfigurationException("ldap.connection.baseDn");
+        // Backward compatibility with the old option
+        if (!StringUtils.isBlank(connection.baseDn)) {
+            connection.getBaseDNs().add(connection.baseDn);
+        }
+
+        if (connection.getBaseDNs().isEmpty()) {
+            throw new ConfigurationException("ldap.connection.baseDNs",
+                    "You must specify at least one Base DN via the singular or plural config option");
         }
 
         if (StringUtils.isBlank(attribute.getUid().getType())) {
@@ -386,7 +403,10 @@ public abstract class LdapConfig {
         log.info("Port: {}", connection.getPort());
         log.info("TLS: {}", connection.isTls());
         log.info("Bind DN: {}", connection.getBindDn());
-        log.info("Base DN: {}", connection.getBaseDn());
+        log.info("Base DNs: {}");
+        for (String baseDN : connection.getBaseDNs()) {
+            log.info("\t- {}", baseDN);
+        }
 
         log.info("Attribute: {}", GsonUtil.get().toJson(attribute));
         log.info("Auth: {}", GsonUtil.get().toJson(auth));

src/main/java/io/kamax/mxisd/config/rest/RestBackendConfig.java

@@ -30,6 +30,8 @@ import org.springframework.context.annotation.Configuration;
 import javax.annotation.PostConstruct;
 import java.net.MalformedURLException;
 import java.net.URL;
+import java.util.Objects;
+import java.util.Optional;
 
 @Configuration
 @ConfigurationProperties("rest")
@@ -58,11 +60,44 @@ public class RestBackendConfig {
 
     }
 
+    public static class ProfileEndpoints {
+
+        private String displayName;
+        private String threepids;
+        private String roles;
+
+        public String getDisplayName() {
+            return displayName;
+        }
+
+        public void setDisplayName(String displayName) {
+            this.displayName = displayName;
+        }
+
+        public String getThreepids() {
+            return threepids;
+        }
+
+        public void setThreepids(String threepids) {
+            this.threepids = threepids;
+        }
+
+        public String getRoles() {
+            return roles;
+        }
+
+        public void setRoles(String roles) {
+            this.roles = roles;
+        }
+
+    }
+
     public static class Endpoints {
 
         private String auth;
         private String directory;
         private IdentityEndpoints identity = new IdentityEndpoints();
+        private ProfileEndpoints profile;
 
         public String getAuth() {
             return auth;
@@ -88,6 +123,14 @@ public class RestBackendConfig {
             this.identity = identity;
         }
 
+        public Optional<ProfileEndpoints> getProfile() {
+            return Optional.ofNullable(profile);
+        }
+
+        public void setProfile(ProfileEndpoints profile) {
+            this.profile = profile;
+        }
+
     }
 
     private Logger log = LoggerFactory.getLogger(RestBackendConfig.class);
@@ -121,21 +164,21 @@ public class RestBackendConfig {
     }
 
     private String buildEndpointUrl(String endpoint) {
-        if (StringUtils.startsWith(endpoint, "/")) {
-            if (StringUtils.isBlank(getHost())) {
-                throw new ConfigurationException("rest.host");
-            }
-
-            try {
-                new URL(getHost());
-            } catch (MalformedURLException e) {
-                throw new ConfigurationException("rest.host", e.getMessage());
-            }
-
-            return getHost() + endpoint;
-        } else {
+        if (!StringUtils.startsWith(endpoint, "/")) {
             return endpoint;
         }
+
+        if (StringUtils.isBlank(getHost())) {
+            throw new ConfigurationException("rest.host");
+        }
+
+        try {
+            new URL(getHost());
+        } catch (MalformedURLException e) {
+            throw new ConfigurationException("rest.host", e.getMessage());
+        }
+
+        return getHost() + endpoint;
     }
 
     @PostConstruct
@@ -149,6 +192,12 @@ public class RestBackendConfig {
             endpoints.identity.setSingle(buildEndpointUrl(endpoints.identity.getSingle()));
             endpoints.identity.setBulk(buildEndpointUrl(endpoints.identity.getBulk()));
 
+            if (Objects.nonNull(endpoints.profile)) {
+                endpoints.profile.setDisplayName(buildEndpointUrl(endpoints.profile.getDisplayName()));
+                endpoints.profile.setThreepids(buildEndpointUrl(endpoints.profile.getThreepids()));
+                endpoints.profile.setRoles(buildEndpointUrl(endpoints.profile.getRoles()));
+            }
+
             log.info("Host: {}", getHost());
             log.info("Auth endpoint: {}", endpoints.getAuth());
             log.info("Directory endpoint: {}", endpoints.getDirectory());

src/main/java/io/kamax/mxisd/controller/app/v1/AppServiceController.java

@@ -20,13 +20,11 @@
 
 package io.kamax.mxisd.controller.app.v1;
 
-import com.google.gson.JsonObject;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.as.AppServiceHandler;
 import io.kamax.mxisd.config.ListenerConfig;
 import io.kamax.mxisd.exception.HttpMatrixException;
 import io.kamax.mxisd.exception.NotAllowedException;
-import io.kamax.mxisd.util.GsonParser;
 import org.apache.commons.lang.StringUtils;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -36,7 +34,8 @@ import org.springframework.web.bind.annotation.*;
 
 import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
-import java.util.List;
+import java.io.IOException;
+import java.util.concurrent.CompletableFuture;
 
 import static org.springframework.web.bind.annotation.RequestMethod.GET;
 import static org.springframework.web.bind.annotation.RequestMethod.PUT;
@@ -50,13 +49,11 @@ public class AppServiceController {
     private final ListenerConfig cfg;
 
     private final String notFoundBody;
-    private final GsonParser parser;
     private final AppServiceHandler handler;
 
     @Autowired
     public AppServiceController(ListenerConfig cfg, AppServiceHandler handler) {
         this.notFoundBody = GsonUtil.get().toJson(GsonUtil.makeObj("errcode", "io.kamax.mxisd.AS_NOT_FOUND"));
-        this.parser = new GsonParser();
 
         this.cfg = cfg;
         this.handler = handler;
@@ -89,23 +86,19 @@ public class AppServiceController {
     }
 
     @RequestMapping(value = "/transactions/{txnId:.+}", method = PUT)
-    public String getTransaction(
+    public CompletableFuture<String> getTransaction(
             HttpServletRequest request,
             @RequestParam(name = "access_token", required = false) String token,
-            @PathVariable String txnId) {
+            @PathVariable String txnId
+    ) {
+        validateToken(token);
+
         try {
-            validateToken(token);
-
-            log.info("Transaction {}: Processing start", txnId);
-            List<JsonObject> events = GsonUtil.asList(GsonUtil.getArray(parser.parse(request.getInputStream()), "events"), JsonObject.class);
-            log.debug("Transaction {}: {} events to process", txnId, events.size());
-            handler.processTransaction(events);
-            log.info("Transaction {}: Processing end", txnId);
-        } catch (Throwable e) {
-            log.error("Unable to properly process transaction {}", txnId, e);
+            log.info("Received AS transaction {}", txnId);
+            return handler.processTransaction(txnId, request.getInputStream());
+        } catch (IOException e) {
+            throw new RuntimeException("AS Transaction " + txnId + ": I/O error when getting input", e);
         }
-
-        return "{}";
     }
 
 }

src/main/java/io/kamax/mxisd/controller/auth/v1/AuthController.java

@@ -20,25 +20,18 @@
 
 package io.kamax.mxisd.controller.auth.v1;
 
-import com.google.gson.*;
-import com.google.i18n.phonenumbers.NumberParseException;
-import com.google.i18n.phonenumbers.PhoneNumberUtil;
-import com.google.i18n.phonenumbers.Phonenumber;
+import com.google.gson.Gson;
+import com.google.gson.JsonElement;
+import com.google.gson.JsonObject;
 import io.kamax.mxisd.auth.AuthManager;
 import io.kamax.mxisd.auth.UserAuthResult;
 import io.kamax.mxisd.controller.auth.v1.io.CredentialsValidationResponse;
-import io.kamax.mxisd.dns.ClientDnsOverwrite;
 import io.kamax.mxisd.exception.JsonMemberNotFoundException;
-import io.kamax.mxisd.exception.RemoteLoginException;
-import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.util.GsonParser;
 import io.kamax.mxisd.util.GsonUtil;
-import io.kamax.mxisd.util.RestClientUtils;
-import org.apache.commons.lang.StringUtils;
+import org.apache.commons.io.IOUtils;
 import org.apache.http.client.methods.CloseableHttpResponse;
 import org.apache.http.client.methods.HttpGet;
-import org.apache.http.client.methods.HttpPost;
-import org.apache.http.client.utils.URIBuilder;
 import org.apache.http.impl.client.CloseableHttpClient;
 import org.apache.http.util.EntityUtils;
 import org.slf4j.Logger;
@@ -54,10 +47,11 @@ import javax.servlet.http.HttpServletRequest;
 import javax.servlet.http.HttpServletResponse;
 import java.io.IOException;
 import java.net.URI;
+import java.nio.charset.StandardCharsets;
 
 @RestController
 @CrossOrigin
-@RequestMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
+@RequestMapping(produces = MediaType.APPLICATION_JSON_VALUE)
 public class AuthController {
 
     // TODO export into SDK
@@ -71,23 +65,9 @@ public class AuthController {
     @Autowired
     private AuthManager mgr;
 
-    @Autowired
-    private LookupStrategy strategy;
-
-    @Autowired
-    private ClientDnsOverwrite dns;
-
     @Autowired
     private CloseableHttpClient client;
 
-    private String resolveProxyUrl(HttpServletRequest req) {
-        URI target = URI.create(req.getRequestURL().toString());
-        URIBuilder builder = dns.transform(target);
-        String urlToLogin = builder.toString();
-        log.info("Proxy resolution: {} to {}", target.toString(), urlToLogin);
-        return urlToLogin;
-    }
-
     @RequestMapping(value = "/_matrix-internal/identity/v1/check_credentials", method = RequestMethod.POST)
     public String checkCredentials(HttpServletRequest req) {
         try {
@@ -120,7 +100,9 @@ public class AuthController {
 
     @RequestMapping(value = logV1Url, method = RequestMethod.GET)
     public String getLogin(HttpServletRequest req, HttpServletResponse res) {
-        try (CloseableHttpResponse hsResponse = client.execute(new HttpGet(resolveProxyUrl(req)))) {
+        URI target = URI.create(req.getRequestURL().toString());
+
+        try (CloseableHttpResponse hsResponse = client.execute(new HttpGet(mgr.resolveProxyUrl(target)))) {
             res.setStatus(hsResponse.getStatusLine().getStatusCode());
             return EntityUtils.toString(hsResponse.getEntity());
         } catch (IOException e) {
@@ -130,98 +112,11 @@ public class AuthController {
 
     @RequestMapping(value = logV1Url, method = RequestMethod.POST)
     public String login(HttpServletRequest req) {
+        URI target = URI.create(req.getRequestURL().toString());
         try {
-            JsonObject reqJsonObject = parser.parse(req.getInputStream());
-
-            // find 3PID in main object
-            GsonUtil.findPrimitive(reqJsonObject, "medium").ifPresent(medium -> {
-                GsonUtil.findPrimitive(reqJsonObject, "address").ifPresent(address -> {
-                    log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
-                    strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
-                        reqJsonObject.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
-                        reqJsonObject.remove("medium");
-                        reqJsonObject.remove("address");
-                    });
-                });
-            });
-
-            // find 3PID in 'identifier' object
-            GsonUtil.findObj(reqJsonObject, "identifier").ifPresent(identifier -> {
-                GsonUtil.findPrimitive(identifier, "type").ifPresent(type -> {
-
-                    if (StringUtils.equals(type.getAsString(), "m.id.thirdparty")) {
-                        GsonUtil.findPrimitive(identifier, "medium").ifPresent(medium -> {
-                            GsonUtil.findPrimitive(identifier, "address").ifPresent(address -> {
-                                log.info("Login request with medium '{}' and address '{}'", medium.getAsString(), address.getAsString());
-                                strategy.findLocal(medium.getAsString(), address.getAsString()).ifPresent(lookupDataOpt -> {
-                                    identifier.addProperty("type", "m.id.user");
-                                    identifier.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
-                                    identifier.remove("medium");
-                                    identifier.remove("address");
-                                });
-                            });
-                        });
-                    }
-
-                    if (StringUtils.equals(type.getAsString(), "m.id.phone")) {
-                        GsonUtil.findPrimitive(identifier, "number").ifPresent(number -> {
-                            GsonUtil.findPrimitive(identifier, "country").ifPresent(country -> {
-                                log.info("Login request with phone '{}'-'{}'", country.getAsString(), number.getAsString());
-                                try {
-                                    PhoneNumberUtil phoneUtil = PhoneNumberUtil.getInstance();
-                                    Phonenumber.PhoneNumber phoneNumber = phoneUtil.parse(number.getAsString(), country.getAsString());
-                                    String canon_phoneNumber = phoneUtil.format(phoneNumber, PhoneNumberUtil.PhoneNumberFormat.E164).replace("+", "");
-                                    String medium = "msisdn";
-                                    strategy.findLocal(medium, canon_phoneNumber).ifPresent(lookupDataOpt -> {
-                                        identifier.addProperty("type", "m.id.user");
-                                        identifier.addProperty("user", lookupDataOpt.getMxid().getLocalPart());
-                                        identifier.remove("country");
-                                        identifier.remove("number");
-                                    });
-                                } catch (NumberParseException e) {
-                                    throw new RuntimeException(e);
-                                }
-                            });
-                        });
-                    }
-                });
-            });
-
-            // invoke 'login' on homeserver
-            HttpPost httpPost = RestClientUtils.post(resolveProxyUrl(req), gson, reqJsonObject);
-            try (CloseableHttpResponse httpResponse = client.execute(httpPost)) {
-                // check http status
-                int status = httpResponse.getStatusLine().getStatusCode();
-                log.info("http status = {}", status);
-                if (status != 200) {
-                    // try to get possible json error message from response
-                    // otherwise just get returned plain error message
-                    String errcode = String.valueOf(httpResponse.getStatusLine().getStatusCode());
-                    String error = EntityUtils.toString(httpResponse.getEntity());
-                    if (httpResponse.getEntity() != null) {
-                        try {
-                            JsonObject bodyJson = new JsonParser().parse(error).getAsJsonObject();
-                            if (bodyJson.has("errcode")) {
-                                errcode = bodyJson.get("errcode").getAsString();
-                            }
-                            if (bodyJson.has("error")) {
-                                error = bodyJson.get("error").getAsString();
-                            }
-                            throw new RemoteLoginException(status, errcode, error, bodyJson);
-                        } catch (JsonSyntaxException e) {
-                            log.warn("Response body is not JSON");
-                        }
-                    }
-                    throw new RemoteLoginException(status, errcode, error);
-                }
-
-                /// return response
-                JsonObject respJsonObject = parser.parseOptional(httpResponse).get();
-                return gson.toJson(respJsonObject);
-            } catch (IOException e) {
-                throw new RuntimeException(e);
-            }
+            return mgr.proxyLogin(target, IOUtils.toString(req.getInputStream(), StandardCharsets.UTF_8));
         } catch (IOException e) {
+            log.error("Unable to read input data from client");
             throw new RuntimeException(e);
         }
     }

src/main/java/io/kamax/mxisd/invitation/InvitationManager.java

@@ -34,7 +34,7 @@ import io.kamax.mxisd.lookup.ThreePidMapping;
 import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.notification.NotificationManager;
 import io.kamax.mxisd.storage.IStorage;
-import io.kamax.mxisd.storage.ormlite.ThreePidInviteIO;
+import io.kamax.mxisd.storage.ormlite.dao.ThreePidInviteIO;
 import org.apache.commons.io.IOUtils;
 import org.apache.commons.lang.RandomStringUtils;
 import org.apache.commons.lang.StringUtils;

src/main/java/io/kamax/mxisd/storage/IStorage.java

@@ -23,8 +23,10 @@ package io.kamax.mxisd.storage;
 import io.kamax.matrix.ThreePid;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.storage.dao.IThreePidSessionDao;
-import io.kamax.mxisd.storage.ormlite.ThreePidInviteIO;
+import io.kamax.mxisd.storage.ormlite.dao.ASTransactionDao;
+import io.kamax.mxisd.storage.ormlite.dao.ThreePidInviteIO;
 
+import java.time.Instant;
 import java.util.Collection;
 import java.util.Optional;
 
@@ -44,4 +46,8 @@ public interface IStorage {
 
     void updateThreePidSession(IThreePidSessionDao session);
 
+    void insertTransactionResult(String localpart, String txnId, Instant completion, String response);
+
+    Optional<ASTransactionDao> getTransactionResult(String localpart, String txnId);
+
 }

src/main/java/io/kamax/mxisd/storage/ormlite/OrmLiteSqliteStorage.java

@@ -31,6 +31,8 @@ import io.kamax.mxisd.exception.InternalServerError;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.dao.IThreePidSessionDao;
+import io.kamax.mxisd.storage.ormlite.dao.ASTransactionDao;
+import io.kamax.mxisd.storage.ormlite.dao.ThreePidInviteIO;
 import io.kamax.mxisd.storage.ormlite.dao.ThreePidSessionDao;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -38,6 +40,7 @@ import org.slf4j.LoggerFactory;
 import java.io.File;
 import java.io.IOException;
 import java.sql.SQLException;
+import java.time.Instant;
 import java.util.ArrayList;
 import java.util.Collection;
 import java.util.List;
@@ -63,17 +66,21 @@ public class OrmLiteSqliteStorage implements IStorage {
 
     private Dao<ThreePidInviteIO, String> invDao;
     private Dao<ThreePidSessionDao, String> sessionDao;
+    private Dao<ASTransactionDao, String> asTxnDao;
 
-    OrmLiteSqliteStorage(String path) {
+    public OrmLiteSqliteStorage(String backend, String path) {
         withCatcher(() -> {
-            File parent = new File(path).getParentFile();
-            if (!parent.mkdirs() && !parent.isDirectory()) {
-                throw new RuntimeException("Unable to create DB parent directory: " + parent);
+            if (path.startsWith("/") && !path.startsWith("//")) {
+                File parent = new File(path).getParentFile();
+                if (!parent.mkdirs() && !parent.isDirectory()) {
+                    throw new RuntimeException("Unable to create DB parent directory: " + parent);
+                }
             }
 
-            ConnectionSource connPool = new JdbcConnectionSource("jdbc:sqlite:" + path);
+            ConnectionSource connPool = new JdbcConnectionSource("jdbc:" + backend + ":" + path);
             invDao = createDaoAndTable(connPool, ThreePidInviteIO.class);
             sessionDao = createDaoAndTable(connPool, ThreePidSessionDao.class);
+            asTxnDao = createDaoAndTable(connPool, ASTransactionDao.class);
         });
     }
 
@@ -178,4 +185,35 @@ public class OrmLiteSqliteStorage implements IStorage {
         });
     }
 
+    @Override
+    public void insertTransactionResult(String localpart, String txnId, Instant completion, String result) {
+        withCatcher(() -> {
+            int created = asTxnDao.create(new ASTransactionDao(localpart, txnId, completion, result));
+            if (created != 1) {
+                throw new RuntimeException("Unexpected row count after DB action: " + created);
+            }
+        });
+    }
+
+    @Override
+    public Optional<ASTransactionDao> getTransactionResult(String localpart, String txnId) {
+        return withCatcher(() -> {
+            ASTransactionDao dao = new ASTransactionDao();
+            dao.setLocalpart(localpart);
+            dao.setTransactionId(txnId);
+            List<ASTransactionDao> daoList = asTxnDao.queryForMatchingArgs(dao);
+
+            if (daoList.size() > 1) {
+                throw new InternalServerError("Lookup for Transaction " +
+                        txnId + " for localpart " + localpart + " returned more than one result");
+            }
+
+            if (daoList.isEmpty()) {
+                return Optional.empty();
+            }
+
+            return Optional.of(daoList.get(0));
+        });
+    }
+
 }

src/main/java/io/kamax/mxisd/storage/ormlite/OrmLiteSqliteStorageBeanFactory.java

@@ -45,13 +45,15 @@ public class OrmLiteSqliteStorageBeanFactory implements FactoryBean<IStorage> {
 
     @PostConstruct
     private void postConstruct() {
-        if (StringUtils.equals("sqlite", storagecfg.getBackend())) {
-            if (StringUtils.isBlank(cfg.getDatabase())) {
-                throw new ConfigurationException("storage.provider.sqlite.database");
-            }
-
-            storage = new OrmLiteSqliteStorage(cfg.getDatabase());
+        if (StringUtils.isBlank(storagecfg.getBackend())) {
+            throw new ConfigurationException("storage.backend");
         }
+
+        if (StringUtils.equals("sqlite", storagecfg.getBackend()) && StringUtils.isBlank(cfg.getDatabase())) {
+            throw new ConfigurationException("storage.provider.sqlite.database");
+        }
+
+        storage = new OrmLiteSqliteStorage(storagecfg.getBackend(), cfg.getDatabase());
     }
 
     @Override

src/main/java/io/kamax/mxisd/storage/ormlite/dao/ASTransactionDao.java

@@ -0,0 +1,86 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.storage.ormlite.dao;
+
+import com.j256.ormlite.field.DatabaseField;
+import com.j256.ormlite.table.DatabaseTable;
+
+import java.time.Instant;
+
+@DatabaseTable(tableName = "as_txn")
+public class ASTransactionDao {
+
+    @DatabaseField(uniqueCombo = true)
+    private String transactionId;
+
+    @DatabaseField(uniqueCombo = true)
+    private String localpart;
+
+    @DatabaseField(canBeNull = false)
+    private long timestamp;
+
+    @DatabaseField(canBeNull = false)
+    private String result;
+
+    public ASTransactionDao() {
+        // Needed for ORMLite
+    }
+
+    public ASTransactionDao(String localpart, String txnId, Instant completion, String result) {
+        setLocalpart(localpart);
+        setTransactionId(txnId);
+        setTimestamp(completion.toEpochMilli());
+        setResult(result);
+    }
+
+    public String getTransactionId() {
+        return transactionId;
+    }
+
+    public void setTransactionId(String transactionId) {
+        this.transactionId = transactionId;
+    }
+
+    public String getLocalpart() {
+        return localpart;
+    }
+
+    public void setLocalpart(String localpart) {
+        this.localpart = localpart;
+    }
+
+    public long getTimestamp() {
+        return timestamp;
+    }
+
+    public void setTimestamp(long timestamp) {
+        this.timestamp = timestamp;
+    }
+
+    public String getResult() {
+        return result;
+    }
+
+    public void setResult(String result) {
+        this.result = result;
+    }
+
+}

src/main/java/io/kamax/mxisd/storage/ormlite/dao/ThreePidInviteIO.java

@@ -18,12 +18,12 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
-package io.kamax.mxisd.storage.ormlite;
+package io.kamax.mxisd.storage.ormlite.dao;
 
-import com.google.gson.Gson;
 import com.google.gson.reflect.TypeToken;
 import com.j256.ormlite.field.DatabaseField;
 import com.j256.ormlite.table.DatabaseTable;
+import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.invitation.IThreePidInviteReply;
 import org.apache.commons.lang.StringUtils;
 
@@ -33,8 +33,6 @@ import java.util.Map;
 @DatabaseTable(tableName = "invite_3pid")
 public class ThreePidInviteIO {
 
-    private static Gson gson = new Gson();
-
     @DatabaseField(id = true)
     private String id;
 
@@ -57,7 +55,7 @@ public class ThreePidInviteIO {
     private String properties;
 
     public ThreePidInviteIO() {
-        // needed for ORMlite
+        // Needed for ORMLite
     }
 
     public ThreePidInviteIO(IThreePidInviteReply data) {
@@ -67,7 +65,7 @@ public class ThreePidInviteIO {
         this.medium = data.getInvite().getMedium();
         this.address = data.getInvite().getAddress();
         this.roomId = data.getInvite().getRoomId();
-        this.properties = gson.toJson(data.getInvite().getProperties());
+        this.properties = GsonUtil.get().toJson(data.getInvite().getProperties());
     }
 
     public String getId() {
@@ -99,7 +97,7 @@ public class ThreePidInviteIO {
             return new HashMap<>();
         }
 
-        return gson.fromJson(properties, new TypeToken<Map<String, String>>() {
+        return GsonUtil.get().fromJson(properties, new TypeToken<Map<String, String>>() {
         }.getType());
     }
 

src/main/java/io/kamax/mxisd/storage/ormlite/dao/ThreePidSessionDao.java

@@ -80,7 +80,7 @@ public class ThreePidSessionDao implements IThreePidSessionDao {
     private boolean isRemoteValidated;
 
     public ThreePidSessionDao() {
-        // stub for ORMLite
+        // Needed for ORMLite
     }
 
     public ThreePidSessionDao(IThreePidSessionDao session) {

src/main/resources/application.yaml

@@ -56,6 +56,10 @@ rest:
     identity:
       single: '/_mxisd/backend/api/v1/identity/lookup/single'
       bulk: '/_mxisd/backend/api/v1/identity/lookup/bulk'
+    profile:
+      displayName: '/_mxisd/backend/api/v1/profile/displayName'
+      threepids: '/_mxisd/backend/api/v1/profile/threepids'
+      roles: '/_mxisd/backend/api/v1/profile/roles'
 
 ldap:
   enabled: false

src/test/java/io/kamax/mxisd/backend/rest/RestProfileProviderTest.java

@@ -0,0 +1,146 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.backend.rest;
+
+import com.github.tomakehurst.wiremock.junit.WireMockRule;
+import io.kamax.matrix.MatrixID;
+import io.kamax.matrix._MatrixID;
+import io.kamax.matrix.json.GsonUtil;
+import io.kamax.mxisd.config.rest.RestBackendConfig;
+import io.kamax.mxisd.exception.InternalServerError;
+import io.kamax.mxisd.profile.JsonProfileRequest;
+import io.kamax.mxisd.profile.JsonProfileResult;
+import org.apache.http.entity.ContentType;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+
+import java.util.Optional;
+
+import static com.github.tomakehurst.wiremock.client.WireMock.*;
+import static io.kamax.mxisd.config.rest.RestBackendConfig.ProfileEndpoints;
+import static org.junit.Assert.*;
+
+public class RestProfileProviderTest {
+
+    @Rule
+    public WireMockRule wireMockRule = new WireMockRule(65000);
+
+    private final String displayNameEndpoint = "/displayName";
+
+    private final _MatrixID userId = MatrixID.from("john", "matrix.localhost").valid();
+
+    private RestProfileProvider p;
+
+    @Before
+    public void before() {
+        ProfileEndpoints endpoints = new ProfileEndpoints();
+        endpoints.setDisplayName(displayNameEndpoint);
+
+        RestBackendConfig cfg = new RestBackendConfig();
+        cfg.setEnabled(true);
+        cfg.setHost("http://localhost:65000");
+        cfg.getEndpoints().setProfile(endpoints);
+        cfg.build();
+
+        p = new RestProfileProvider(cfg);
+    }
+
+    @Test
+    public void forNameFound() {
+        String value = "This is my display name";
+
+        JsonProfileResult r = new JsonProfileResult();
+        r.setDisplayName(value);
+        stubFor(post(urlEqualTo(displayNameEndpoint))
+                .willReturn(aResponse()
+                        .withHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType())
+                        .withBody(GsonUtil.get().toJson(GsonUtil.makeObj("profile", r)))
+                )
+        );
+
+        Optional<String> v = p.getDisplayName(userId);
+
+        verify(postRequestedFor(urlMatching(displayNameEndpoint))
+                .withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
+                .withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
+        );
+
+        assertTrue(v.isPresent());
+        assertEquals(value, v.get());
+    }
+
+    @Test
+    public void forNameNotFound() {
+        stubFor(post(urlEqualTo(displayNameEndpoint))
+                .willReturn(aResponse()
+                        .withStatus(404)
+                )
+        );
+
+        Optional<String> v = p.getDisplayName(userId);
+
+        verify(postRequestedFor(urlMatching(displayNameEndpoint))
+                .withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
+                .withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
+        );
+
+        assertFalse(v.isPresent());
+    }
+
+    @Test
+    public void forNameEmptyBody() {
+        stubFor(post(urlEqualTo(displayNameEndpoint))
+                .willReturn(aResponse()
+                        .withHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType())
+                )
+        );
+
+        Optional<String> v = p.getDisplayName(userId);
+
+        verify(postRequestedFor(urlMatching(displayNameEndpoint))
+                .withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
+                .withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
+        );
+
+        assertFalse(v.isPresent());
+    }
+
+    @Test(expected = InternalServerError.class)
+    public void forNameInvalidBody() {
+        stubFor(post(urlEqualTo(displayNameEndpoint))
+                .willReturn(aResponse()
+                        .withHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType())
+                        .withBody("This is not a valid JSON object")
+                )
+        );
+
+        try {
+            p.getDisplayName(userId);
+        } finally {
+            verify(postRequestedFor(urlMatching(displayNameEndpoint))
+                    .withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
+                    .withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
+            );
+        }
+    }
+
+}

src/test/java/io/kamax/mxisd/test/backend/ldap/LdapAuthTest.java

@@ -0,0 +1,115 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.test.backend.ldap;
+
+import com.unboundid.ldap.listener.InMemoryDirectoryServer;
+import com.unboundid.ldap.listener.InMemoryDirectoryServerConfig;
+import com.unboundid.ldap.listener.InMemoryListenerConfig;
+import com.unboundid.ldap.sdk.LDAPException;
+import io.kamax.matrix.MatrixID;
+import io.kamax.mxisd.auth.provider.BackendAuthResult;
+import io.kamax.mxisd.backend.ldap.LdapAuthProvider;
+import io.kamax.mxisd.config.MatrixConfig;
+import io.kamax.mxisd.config.ldap.LdapConfig;
+import io.kamax.mxisd.config.ldap.generic.GenericLdapConfig;
+import org.junit.AfterClass;
+import org.junit.BeforeClass;
+import org.junit.Test;
+
+import java.util.ArrayList;
+
+import static org.junit.Assert.assertFalse;
+
+public class LdapAuthTest {
+
+    private static InMemoryDirectoryServer ds;
+    private static ArrayList<String> dnList = new ArrayList<>();
+
+    @BeforeClass
+    public static void beforeClass() throws LDAPException {
+        dnList.add("dc=1,dc=mxisd,dc=example,dc=org");
+        dnList.add("dc=2,dc=mxisd,dc=example,dc=org");
+        dnList.add("dc=3,dc=mxisd,dc=example,dc=org");
+
+        InMemoryListenerConfig lCfg = InMemoryListenerConfig.createLDAPConfig("localhost", 65001);
+        InMemoryDirectoryServerConfig config =
+                new InMemoryDirectoryServerConfig(dnList.get(0), dnList.get(1), dnList.get(2));
+        config.addAdditionalBindCredentials("cn=mxisd", "mxisd");
+        config.setListenerConfigs(lCfg);
+
+        ds = new InMemoryDirectoryServer(config);
+        ds.startListening();
+    }
+
+    @AfterClass
+    public static void afterClass() {
+        ds.shutDown(true);
+    }
+
+    @Test
+    public void singleDn() {
+        MatrixConfig mxCfg = new MatrixConfig();
+        mxCfg.setDomain("example.org");
+        mxCfg.build();
+
+        LdapConfig cfg = new GenericLdapConfig();
+        cfg.getConnection().setHost("localhost");
+        cfg.getConnection().setPort(65001);
+        cfg.getConnection().setBaseDn(dnList.get(0));
+        cfg.getConnection().setBindDn("cn=mxisd");
+        cfg.getConnection().setBindPassword("mxisd");
+
+        LdapConfig.UID uid = new LdapConfig.UID();
+        uid.setType("uid");
+        uid.setValue("saMAccountName");
+        cfg.getAttribute().setUid(uid);
+        cfg.build();
+
+        LdapAuthProvider p = new LdapAuthProvider(cfg, mxCfg);
+        BackendAuthResult result = p.authenticate(MatrixID.from("john", "example.org").valid(), "doe");
+        assertFalse(result.isSuccess());
+    }
+
+    @Test
+    public void multiDNs() {
+        MatrixConfig mxCfg = new MatrixConfig();
+        mxCfg.setDomain("example.org");
+        mxCfg.build();
+
+        LdapConfig cfg = new GenericLdapConfig();
+        cfg.getConnection().setHost("localhost");
+        cfg.getConnection().setPort(65001);
+        cfg.getConnection().setBaseDNs(dnList);
+        cfg.getConnection().setBindDn("cn=mxisd");
+        cfg.getConnection().setBindPassword("mxisd");
+
+        LdapConfig.UID uid = new LdapConfig.UID();
+        uid.setType("uid");
+        uid.setValue("saMAccountName");
+        cfg.getAttribute().setUid(uid);
+        cfg.build();
+
+        LdapAuthProvider p = new LdapAuthProvider(cfg, mxCfg);
+        BackendAuthResult result = p.authenticate(MatrixID.from("john", "example.org").valid(), "doe");
+        assertFalse(result.isSuccess());
+    }
+
+}

src/test/java/io/kamax/mxisd/test/storage/OrmLiteSqliteStorageTest.java

@@ -0,0 +1,44 @@
+/*
+ * mxisd - Matrix Identity Server Daemon
+ * Copyright (C) 2018 Kamax Sarl
+ *
+ * https://www.kamax.io/
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU Affero General Public License as
+ * published by the Free Software Foundation, either version 3 of the
+ * License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Affero General Public License for more details.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+package io.kamax.mxisd.test.storage;
+
+import io.kamax.mxisd.storage.ormlite.OrmLiteSqliteStorage;
+import org.junit.Test;
+
+import java.time.Instant;
+
+public class OrmLiteSqliteStorageTest {
+
+    @Test
+    public void insertAsTxnDuplicate() {
+        OrmLiteSqliteStorage store = new OrmLiteSqliteStorage("sqlite", ":memory:");
+        store.insertTransactionResult("mxisd", "1", Instant.now(), "{}");
+        store.insertTransactionResult("mxisd", "2", Instant.now(), "{}");
+    }
+
+    @Test(expected = RuntimeException.class)
+    public void insertAsTxnSame() {
+        OrmLiteSqliteStorage store = new OrmLiteSqliteStorage("sqlite", ":memory:");
+        store.insertTransactionResult("mxisd", "1", Instant.now(), "{}");
+        store.insertTransactionResult("mxisd", "1", Instant.now(), "{}");
+    }
+
+}