Restart mxisd service on Debian package upgrade/install if possible

- Support 3PID unbind via 3PID sessions

README.md

@@ -14,7 +14,7 @@ mxisd - Federated Matrix Identity Server
 
 # Overview
 mxisd is a Federated Matrix Identity server for self-hosted Matrix infrastructures with [enhanced features](#features).
-As an enhanced Identity service, it implements the [Identity service API](https://matrix.org/docs/spec/identity_service/r0.1.0.html)
+As an enhanced Identity service, it implements the [Identity service API](https://matrix.org/docs/spec/identity_service/r0.2.0.html)
 and several [extra features](#features) that greatly enhance user experience within Matrix.
 It is the one stop shop for anything regarding Authentication, Directory and Identity management in Matrix built in a
 single coherent product.
@@ -34,15 +34,15 @@ users. 3PIDs can be anything that uniquely and globally identify a user, like:
 If you are unfamiliar with the Identity vocabulary and concepts in Matrix, **please read this [introduction](docs/concepts.md)**.  
 
 # Features
-[Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://matrix.org/docs/spec/identity_service/r0.1.0.html#general-principles):
+[Identity](docs/features/identity.md): As a [regular Matrix Identity service](https://matrix.org/docs/spec/identity_service/r0.2.0.html#general-principles):
 - Search for people by 3PID using its own Identity stores
-  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#association-lookup))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.2.0.html#association-lookup))
 - Invite people to rooms by 3PID using its own Identity stores, with notifications to the invitee (Email, SMS, etc.)
-  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#post-matrix-identity-api-v1-store-invite))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.2.0.html#invitation-storage))
-- Allow users to add 3PIDs to their settings/profile
+- Allow users to add/remove 3PIDs to their settings/profile via 3PID sessions
-  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#establishing-associations))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.2.0.html#establishing-associations))
 - Register accounts on your Homeserver with 3PIDs
-  ([Spec](https://matrix.org/docs/spec/identity_service/r0.1.0.html#establishing-associations))
+  ([Spec](https://matrix.org/docs/spec/identity_service/r0.2.0.html#establishing-associations))
 
 As an enhanced Identity service:
 - [Federation](docs/features/federation.md): Use a recursive lookup mechanism when searching and inviting people by 3PID,

build.gradle

@@ -229,6 +229,12 @@ task debBuild(dependsOn: shadowJar) {
                 value: debDataPath
         )
 
+        ant.replace(
+                file: "${debBuildDebianPath}/postinst",
+                token: '%DEB_CONF_FILE%',
+                value: "${debConfPath}/mxisd.yaml"
+        )
+
         ant.chmod(
                 file: "${debBuildDebianPath}/postinst",
                 perm: 'a+x'

docs/features/experimental/application-service.md

@@ -1,6 +1,6 @@
 # Application Service
 **WARNING:** These features are currently highly experimental. They can be removed or modified without notice.  
-All the features requires a Homeserver capable of connecting [Application Services](https://matrix.org/docs/spec/application_service/r0.1.0.html).
+All the features requires a Homeserver capable of connecting [Application Services](https://matrix.org/docs/spec/application_service/r0.1.1.html).
 
 The following capabilities are provided in this feature:
 - [Admin commands](#admin-commands)

docs/features/identity.md

@@ -1,5 +1,5 @@
 # Identity
-Implementation of the [Identity Service API r0.1.0](https://matrix.org/docs/spec/identity_service/r0.1.0.html).
+Implementation of the [Identity Service API r0.2.0](https://matrix.org/docs/spec/identity_service/r0.2.0.html).
 
 - [Lookups](#lookups)
 - [Invitations](#invitations)

docs/features/registration.md

@@ -3,7 +3,7 @@
 - [Integration](#integration)
   - [Reverse Proxy](#reverse-proxy)
     - [nginx](#nginx)
-    - [Apache](#apache)
+    - [Apache2](#apache2)
   - [Homeserver](#homeserver)
     - [synapse](#synapse)
 - [Configuration](#configuration)
@@ -16,7 +16,7 @@
 Registration is an enhanced feature of mxisd to control registrations involving 3PIDs on a Homeserver based on policies:
 - Match pending 3PID invites on the server
 - Match 3PID pattern, like a specific set of domains for emails
-- In futher releases, use 3PIDs found in Identity stores
+- In further releases, use 3PIDs found in Identity stores
 
 It aims to help open or invite-only registration servers control what is possible to do and ensure only approved people
 can register on a given server in a implementation-agnostic manner.
@@ -36,14 +36,14 @@ Later version(s) of this feature may directly control registration itself to cre
 ### Reverse Proxy
 #### nginx
 ```nginx
-location ^/_matrix/client/r0/register/[^/]/?$ {
+location ~* ^/_matrix/client/r0/register/[^/]+/requestToken$ {
-	proxy_pass		http://127.0.0.1:8090;
+	proxy_pass		http://localhost:8090;
 	proxy_set_header	Host $host;
 	proxy_set_header	X-Forwarded-For $remote_addr;
 }
 ```
 
-#### apache
+#### Apache2
 > TBC
 
 ### Homeserver
@@ -55,8 +55,8 @@ registrations_require_3pid:
 ```
 
 ## Configuration
-See the [Configuration](../configuration.md) introduction doc on how to read the configuration keys.  
+See the [Configuration](../configure.md) introduction doc on how to read the configuration keys.  
-An example of working configuration is avaiable at the end of this section.
+An example of working configuration is available at the end of this section.
 ### Enable/Disable
 `register.allowed`, taking a boolean, can be used to enable/disable registration if the attempt is not 3PID-based.  
 `false` is the default value to prevent open registration, as you must allow it on the homeserver side.
@@ -72,7 +72,7 @@ At this time, only `email` is supported with 3PID specific configuration with th
 **Base key**: `register.threepid.email`
 
 ##### Domain whitelist/blacklist
-If you would like to control which domains are allowed to be used when registrating with an email, the following sub-keys
+If you would like to control which domains are allowed to be used when registering with an email, the following sub-keys
 are available:
 - `domain.whitelist`
 - `domain.blacklist`
@@ -82,7 +82,7 @@ The value format is an hybrid between glob patterns and postfix configuration fi
 - `.<domain>` will only match sub-domain(s)
 - `<domain>` will only match the exact domain
 
-The following table illustrates pattern and maching status against example values:
+The following table illustrates pattern and matching status against example values:
 
 | Config value   | Matches `example.org` | Matches `sub.example.org` |
 |--------------- |-----------------------|---------------------------|

docs/threepids/session/session-views.md

@@ -26,6 +26,14 @@ Two configuration keys are available that accept paths to HTML templates:
 - `success`
 - `failure`
 
+### Serving static assets
+mxisd will not serve any static asset (images, JS, CSS, etc.). If such are needed, you will need to serve them using the
+reverse proxy sitting in front of mxisd using a path outside of the `/_matrix/identity/` namespace. We advise using
+the base path `/static/` for such use cases, allowing to remain under the same hostname/origin.
+
+You can also serve such assets using absolute URL, possibly under other domains, but be aware of Cross-Origin restrictions
+in browsers which are out of scope of mxisd.
+
 ## Placeholders
 ### Success
 No object/placeholder are currently available.

src/debian/postinst

@@ -11,3 +11,9 @@ ln -sfT /usr/lib/mxisd/mxisd /usr/bin/mxisd
 
 # Enable systemd service
 systemctl enable mxisd.service
+
+# If we already have a config file setup, we attempt to run mxisd automatically
+# Specifically targeted at upgrades where the service needs to be restarted
+if [ -f "%DEB_CONF_FILE%" ]; then
+    systemctl restart mxisd.service
+fi

src/main/java/io/kamax/mxisd/Mxisd.java

@@ -118,7 +118,7 @@ public class Mxisd {
         idStrategy = new RecursivePriorityLookupStrategy(cfg.getLookup(), ThreePidProviders.get(), bridgeFetcher);
         pMgr = new ProfileManager(ProfileProviders.get(), clientDns, httpClient);
         notifMgr = new NotificationManager(cfg.getNotification(), NotificationHandlers.get());
-        sessMgr = new SessionManager(cfg.getSession(), cfg.getMatrix(), store, notifMgr, idStrategy);
+        sessMgr = new SessionManager(cfg.getSession(), cfg.getMatrix(), store, notifMgr);
         invMgr = new InvitationManager(cfg, store, idStrategy, keyMgr, signMgr, resolver, notifMgr, pMgr);
         authMgr = new AuthManager(cfg, AuthProviders.get(), idStrategy, invMgr, clientDns, httpClient);
         dirMgr = new DirectoryManager(cfg.getDirectory(), clientDns, httpClient, DirectoryProviders.get());

src/main/java/io/kamax/mxisd/crypto/KeyType.java

@@ -22,7 +22,7 @@ package io.kamax.mxisd.crypto;
 
 /**
  * Types of keys used by an Identity server.
- * See https://matrix.org/docs/spec/identity_service/r0.1.0.html#key-management
+ * See https://matrix.org/docs/spec/identity_service/r0.2.0.html#key-management
  */
 public enum KeyType {
 

src/main/java/io/kamax/mxisd/http/undertow/handler/identity/v1/SessionTpidUnbindHandler.java

@@ -21,15 +21,22 @@
 package io.kamax.mxisd.http.undertow.handler.identity.v1;
 
 import com.google.gson.JsonObject;
+import io.kamax.mxisd.exception.BadRequestException;
+import io.kamax.mxisd.exception.NotAllowedException;
 import io.kamax.mxisd.http.IsAPIv1;
 import io.kamax.mxisd.http.undertow.handler.BasicHttpHandler;
 import io.kamax.mxisd.session.SessionManager;
 import io.undertow.server.HttpServerExchange;
+import org.apache.commons.lang3.StringUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 public class SessionTpidUnbindHandler extends BasicHttpHandler {
 
     public static final String Path = IsAPIv1.Base + "/3pid/unbind";
 
+    private static final Logger log = LoggerFactory.getLogger(SessionTpidUnbindHandler.class);
+
     private final SessionManager sessionMgr;
 
     public SessionTpidUnbindHandler(SessionManager sessionMgr) {
@@ -38,6 +45,18 @@ public class SessionTpidUnbindHandler extends BasicHttpHandler {
 
     @Override
     public void handleRequest(HttpServerExchange exchange) {
+        String auth = exchange.getRequestHeaders().getFirst("Authorization");
+        if (StringUtils.isNotEmpty(auth)) {
+            // We have a auth header to process
+            if (StringUtils.startsWith(auth, "X-Matrix ")) {
+                log.warn("A remote host attempted to unbind without proper authorization. Request was denied");
+                log.warn("See https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy for more info");
+                throw new NotAllowedException("3PID can only be removed via 3PID sessions, not via Homeserver signature");
+            } else {
+                throw new BadRequestException("Illegal authorization type");
+            }
+        }
+
         JsonObject body = parseJsonObject(exchange);
         sessionMgr.unbind(body);
         writeBodyAsUtf8(exchange, "{}");

src/main/java/io/kamax/mxisd/session/SessionManager.java

@@ -27,14 +27,13 @@ import io.kamax.matrix._MatrixID;
 import io.kamax.matrix.json.GsonUtil;
 import io.kamax.mxisd.config.MatrixConfig;
 import io.kamax.mxisd.config.SessionConfig;
+import io.kamax.mxisd.exception.BadRequestException;
 import io.kamax.mxisd.exception.NotAllowedException;
-import io.kamax.mxisd.exception.NotImplementedException;
 import io.kamax.mxisd.exception.SessionNotValidatedException;
 import io.kamax.mxisd.exception.SessionUnknownException;
 import io.kamax.mxisd.lookup.SingleLookupReply;
 import io.kamax.mxisd.lookup.SingleLookupRequest;
 import io.kamax.mxisd.lookup.ThreePidValidation;
-import io.kamax.mxisd.lookup.strategy.LookupStrategy;
 import io.kamax.mxisd.notification.NotificationManager;
 import io.kamax.mxisd.storage.IStorage;
 import io.kamax.mxisd.storage.dao.IThreePidSessionDao;
@@ -56,20 +55,17 @@ public class SessionManager {
     private MatrixConfig mxCfg;
     private IStorage storage;
     private NotificationManager notifMgr;
-    private LookupStrategy lookupMgr;
 
     public SessionManager(
             SessionConfig cfg,
             MatrixConfig mxCfg,
             IStorage storage,
-            NotificationManager notifMgr,
+            NotificationManager notifMgr
-            LookupStrategy lookupMgr
     ) {
         this.cfg = cfg;
         this.mxCfg = mxCfg;
         this.storage = storage;
         this.notifMgr = notifMgr;
-        this.lookupMgr = lookupMgr;
     }
 
     private ThreePidSession getSession(String sid, String secret) {
@@ -179,53 +175,32 @@ public class SessionManager {
     }
 
     public void unbind(JsonObject reqData) {
-        if (reqData.entrySet().size() == 2 && reqData.has("mxid") && reqData.has("threepid")) {
+        _MatrixID mxid;
-            /* This is a HS request to remove a 3PID and is considered:
-             * - An attack on user privacy
-             * - A baffling spec breakage requiring IS and HS 3PID info to be independent [1]
-             * - A baffling spec breakage that 3PID (un)bind is only one way [2]
-             *
-             * Given the lack of response on our extensive feedback on the proposal [3] which has not landed in the spec yet [4],
-             * We'll be denying such unbind requests and will inform users using their 3PID that a fraudulent attempt of
-             * removing their 3PID binding has been attempted and blocked.
-             *
-             * [1]: https://matrix.org/docs/spec/client_server/r0.4.0.html#adding-account-administrative-contact-information
-             * [2]: https://matrix.org/docs/spec/identity_service/r0.1.0.html#privacy
-             * [3]: https://docs.google.com/document/d/135g2muVxmuml0iUnLoTZxk8M2ZSt3kJzg81chGh51yg/edit
-             * [4]: https://github.com/matrix-org/matrix-doc/issues/1194
-             */
-
-            log.warn("A remote host attempted to unbind without proper authorization. Request was denied");
-            log.warn("See https://github.com/kamax-matrix/mxisd/wiki/mxisd-and-your-privacy for more info");
-
-            if (!cfg.getPolicy().getUnbind().getFraudulent().getSendWarning()) {
-                log.info("Not sending notification to 3PID owner as per configuration");
-            } else {
-                log.info("Sending notification to 3PID owner as per configuration");
-
-                ThreePid tpid = GsonUtil.get().fromJson(GsonUtil.getObj(reqData, "threepid"), ThreePid.class);
-                Optional<SingleLookupReply> lookup = lookupMgr.findLocal(tpid.getMedium(), tpid.getAddress());
-                if (!lookup.isPresent()) {
-                    log.info("No 3PID owner found, not sending any notification");
-                } else {
-                    log.info("3PID owner found, sending notification");
         try {
-                        notifMgr.sendForFraudulentUnbind(tpid);
+            mxid = MatrixID.asAcceptable(GsonUtil.getStringOrThrow(reqData, "mxid"));
-                        log.info("Notification sent");
+        } catch (IllegalArgumentException e) {
-                    } catch (NotImplementedException e) {
+            throw new BadRequestException(e.getMessage());
-                        log.warn("Unable to send notification: {}", e.getMessage());
-                    } catch (RuntimeException e) {
-                        log.warn("Unable to send notification due to unknown error. See stacktrace below", e);
-                    }
-                }
         }
 
-            throw new NotAllowedException("You have attempted to alter 3PID bindings, which can only be done by the 3PID owner directly. " +
+        String sid = GsonUtil.getStringOrNull(reqData, "sid");
-                    "We have informed the 3PID owner of your fraudulent attempt.");
+        String secret = GsonUtil.getStringOrNull(reqData, "client_secret");
+        ThreePid tpid = GsonUtil.get().fromJson(GsonUtil.getObj(reqData, "threepid"), ThreePid.class);
+
+        // We ensure the session was validated
+        ThreePidSession session = getSessionIfValidated(sid, secret);
+
+        // As per spec, we can only allow if the provided 3PID matches the validated session
+        if (!session.getThreePid().equals(tpid)) {
+            throw new BadRequestException("3PID to unbind does not match the one from the validated session");
         }
 
-        log.info("Denying unbind request as the endpoint is not defined in the spec.");
+        // We only allow unbind for the domain we manage, mirroring bind
-        throw new NotAllowedException(499, "This endpoint does not exist in the spec and therefore is not supported.");
+        if (!StringUtils.equalsIgnoreCase(mxCfg.getDomain(), mxid.getDomain())) {
+            throw new NotAllowedException("Only Matrix IDs from domain " + mxCfg.getDomain() + " can be unbound");
+        }
+
+        log.info("Session {}: Unbinding of {}:{} to Matrix ID {} is accepted",
+                session.getId(), session.getThreePid().getMedium(), session.getThreePid().getAddress(), mxid.getId());
     }
 
 }