cqrenet/mxids
3
0
Fork 0
Code Pull Requests 10 Actions Releases Activity

Add support for Profile feature in REST Identity store (Fix #91)

Browse Source
This commit is contained in:
Max Dor
2018-12-21 19:21:15 +01:00
parent ad1b91f370
commit 92cf5c6b21
7 changed files with 462 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
docs/features/experimental/profile.md
View File

@@ -1,12 +1,16 @@
# Profile enhancement
**WARNING**: Alpha feature, not officially supported. Do not use.
# Profile
**WARNING**: The following sub-features are considered experimental and not officially supported. Use at your own peril.
This feature allows to enhance a profile query with more info than just Matrix ID and Display name, allowing for custom
applications to retrieve custom data not currently provided by synapse, per example.
## Public Profile enhancement
This feature allows to enhance a public profile query with more info than just Matrix ID and Display name, allowing for
custom applications to retrieve custom data not currently provided by synapse, per example.
## Configuration
### Reverse proxy
#### Apache
**WARNING**: This information can be queried without authentication as per the specification. Do not enable unless in a
controlled environment.
### Configuration
#### Reverse proxy
##### Apache
```apache
ProxyPassMatch "^/_matrix/client/r0/profile/([^/]+)$" "http://127.0.0.1:8090/_matrix/client/r0/profile/$1"
```

10
docs/features/profile.md Normal file
View File

@@ -0,0 +1,10 @@
# Profile
The profile feature does not do anything on its own and acts as a support feature for others, allowing to retrieve
information about a user based on its Matrix ID by querying enabled [Identity stores](../stores/README.md).
Currently supported:
- Display name
- 3PIDs
- Roles/Groups
Experimental sub-features are also available. See [the dedicated document](experimental/profile.md).

111
docs/stores/rest.md
View File

@@ -3,38 +3,45 @@ The REST backend allows you to query identity data in existing webapps, like:
- Forums (phpBB, Discourse, etc.)
- Custom Identity stores (Keycloak, ...)
- CRMs (Wordpress, ...)
- self-hosted clouds (Nextcloud, ownCloud, ...)
- Self-hosted clouds (Nextcloud, ownCloud, ...)
To integrate this backend with your webapp, you will need to implement three specific REST endpoints detailed below.
To integrate this backend with your webapp, you will need to implement the REST endpoints described below.
## Features
| Name | Supported? |
|----------------|------------|
| Authentication | Yes |
| Directory | Yes |
| Identity | Yes |
| Profile | No |
| Name | Supported? |
|-------------------------------------------------|------------|
| [Authentication](../features/authentication.md) | Yes |
| [Directory](../features/directory.md) | Yes |
| [Identity](../features/identity.md) | Yes |
| [Profile](../features/profile.md) | Yes |
## Configuration
| Key | Default | Description |
|----------------------------------|------------------------------------------------|------------------------------------------------------|
| `rest.enabled` | `false` | Globally enable/disable the REST backend |
| `rest.host` | *None* | Default base URL to use for the different endpoints. |
| `rest.endpoints.auth` | `/_mxisd/backend/api/v1/auth/login` | Validate credentials and get user profile |
| `rest.endpoints.directory` | `/_mxisd/backend/api/v1/directory/user/search` | Search for users by arbitrary input |
| `rest.endpoints.identity.single` | `/_mxisd/backend/api/v1/identity/single` | Endpoint to query a single 3PID |
| `rest.endpoints.identity.bulk` | `/_mxisd/backend/api/v1/identity/bulk` | Endpoint to query a list of 3PID |
| Key | Default | Description |
|--------------------------------------|------------------------------------------------|------------------------------------------------------|
| `rest.enabled` | `false` | Globally enable/disable the REST backend |
| `rest.host` | *None* | Default base URL to use for the different endpoints. |
| `rest.endpoints.auth` | `/_mxisd/backend/api/v1/auth/login` | Validate credentials and get user profile |
| `rest.endpoints.directory` | `/_mxisd/backend/api/v1/directory/user/search` | Search for users by arbitrary input |
| `rest.endpoints.identity.single` | `/_mxisd/backend/api/v1/identity/single` | Endpoint to query a single 3PID |
| `rest.endpoints.identity.bulk` | `/_mxisd/backend/api/v1/identity/bulk` | Endpoint to query a list of 3PID |
| `rest.endpoints.profile.displayName` | `/_mxisd/backend/api/v1/profile/displayName` | Query the display name for a Matrix ID
| `rest.endpoints.profile.threepids` | `/_mxisd/backend/api/v1/profile/threepids` | Query the 3PIDs for a Matrix ID
| `rest.endpoints.profile.roles` | `/_mxisd/backend/api/v1/profile/roles` | Query the Roles for a Matrix ID
Endpoint values can handle two formats:
- URL Path starting with `/` that gets happened to the `rest.host`
- Full URL, if you want each endpoint to go to a specific server/protocol/port
If an endpoint value is configured as an empty string, it will disable that specific feature, essentially bypassing the
Identity store for that specific query.
`rest.host` is mandatory if at least one endpoint is not a full URL.
## Endpoints
### Authentication
HTTP method: `POST`
Content-type: JSON UTF-8
- Method: `POST`
- Content-Type: `application/json` (JSON)
- Encoding: `UTF8`
#### Request Body
```json
@@ -87,8 +94,9 @@ If the authentication succeed:
```
### Directory
HTTP method: `POST`
Content-type: JSON UTF-8
- Method: `POST`
- Content-Type: `application/json` (JSON)
- Encoding: `UTF8`
#### Request Body
```json
@@ -113,7 +121,7 @@ If users found:
"user_id": "UserIdLocalpart"
},
{
...
"...": "..."
}
]
}
@@ -129,10 +137,11 @@ If no user found:
### Identity
#### Single 3PID lookup
HTTP method: `POST`
Content-type: JSON UTF-8
- Method: `POST`
- Content-Type: `application/json` (JSON)
- Encoding: `UTF8`
#### Request Body
##### Request Body
```json
{
"lookup": {
@@ -142,7 +151,7 @@ Content-type: JSON UTF-8
}
```
#### Response Body
##### Response Body
If a match was found:
- `lookup.id.type` supported values: `localpart`, `mxid`
```json
@@ -164,10 +173,11 @@ If no match was found:
```
#### Bulk 3PID lookup
HTTP method: `POST`
Content-type: JSON UTF-8
- Method: `POST`
- Content-Type: `application/json` (JSON)
- Encoding: `UTF8`
#### Request Body
##### Request Body
```json
{
"lookup": [
@@ -183,7 +193,7 @@ Content-type: JSON UTF-8
}
```
#### Response Body
##### Response Body
For all entries where a match was found:
- `lookup[].id.type` supported values: `localpart`, `mxid`
```json
@@ -215,3 +225,46 @@ If no match was found:
"lookup": []
}
```
### Profile
#### Request Body
For all requests, the values are the same:
- Method: `POST`
- Content-Type: `application/json` (JSON)
- Encoding: `UTF8`
With body (example values):
##### Request Body
```json
{
"mxid": "@john.doe:example.org",
"localpart": "john.doe",
"domain": "example.org"
}
```
#### Response Body
For all responses, the same object structure will be parsed, making the non-relevant fields as optional.
Structure with example values:
```json
{
"profile": {
"display_name": "John Doe",
"threepids": [
{
"medium": "email",
"address": "john.doe@example.org"
},
{
"...": "..."
}
],
"roles": [
"DomainUsers",
"SalesOrg",
"..."
]
}
}
```
The base `profile` key is mandatory. `display_name`, `threepids` and `roles` are only to be returned on the relevant request.

147
src/main/java/io/kamax/mxisd/backend/rest/RestProfileProvider.java Normal file
View File

@@ -0,0 +1,147 @@
/*
* mxisd - Matrix Identity Server Daemon
* Copyright (C) 2018 Kamax Sarl
*
* https://www.kamax.io/
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package io.kamax.mxisd.backend.rest;
import com.google.gson.JsonObject;
import com.google.gson.JsonSyntaxException;
import io.kamax.matrix._MatrixID;
import io.kamax.matrix._ThreePid;
import io.kamax.matrix.json.GsonUtil;
import io.kamax.matrix.json.InvalidJsonException;
import io.kamax.mxisd.config.rest.RestBackendConfig;
import io.kamax.mxisd.exception.InternalServerError;
import io.kamax.mxisd.profile.JsonProfileRequest;
import io.kamax.mxisd.profile.JsonProfileResult;
import io.kamax.mxisd.profile.ProfileProvider;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang.StringUtils;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import java.io.IOException;
import java.net.URISyntaxException;
import java.nio.charset.StandardCharsets;
import java.util.*;
import java.util.function.Function;
@Component
public class RestProfileProvider extends RestProvider implements ProfileProvider {
private final Logger log = LoggerFactory.getLogger(RestProfileProvider.class);
public RestProfileProvider(RestBackendConfig cfg) {
super(cfg);
}
@Override
public boolean isEnabled() {
return cfg.isEnabled() && cfg.getEndpoints().getProfile().isPresent();
}
private <T> Optional<T> doRequest(
_MatrixID userId,
Function<RestBackendConfig.ProfileEndpoints, Optional<String>> endpoint,
Function<JsonProfileResult, Optional<T>> value
) {
return cfg.getEndpoints().getProfile()
// We get the endpoint
.flatMap(endpoint)
// We only continue if there is a value
.filter(StringUtils::isNotBlank)
// We use the endpoint
.flatMap(url -> {
try {
URIBuilder builder = new URIBuilder(url);
HttpPost req = new HttpPost(builder.build());
req.setEntity(new StringEntity(GsonUtil.get().toJson(new JsonProfileRequest(userId)), ContentType.APPLICATION_JSON));
try (CloseableHttpResponse res = client.execute(req)) {
int sc = res.getStatusLine().getStatusCode();
if (sc == 404) {
log.info("Got 404 - No result found");
return Optional.empty();
}
if (sc != 200) {
throw new InternalServerError("Unexpected backed status code: " + sc);
}
String body = IOUtils.toString(res.getEntity().getContent(), StandardCharsets.UTF_8);
if (StringUtils.isBlank(body)) {
log.warn("Backend response body is empty/blank, expected JSON object with profile key");
return Optional.empty();
}
Optional<JsonObject> pJson = GsonUtil.findObj(GsonUtil.parseObj(body), "profile");
if (!pJson.isPresent()) {
log.warn("Backend response body is invalid, expected JSON object with profile key");
return Optional.empty();
}
JsonProfileResult profile = gson.fromJson(pJson.get(), JsonProfileResult.class);
return value.apply(profile);
}
} catch (JsonSyntaxException | InvalidJsonException e) {
log.error("Unable to parse backend response as JSON", e);
throw new InternalServerError(e);
} catch (URISyntaxException e) {
log.error("Unable to build a valid request URL", e);
throw new InternalServerError(e);
} catch (IOException e) {
log.error("I/O Error during backend request", e);
throw new InternalServerError();
}
});
}
@Override
public Optional<String> getDisplayName(_MatrixID userId) {
return doRequest(userId, p -> Optional.ofNullable(p.getDisplayName()), profile -> Optional.ofNullable(profile.getDisplayName()));
}
@Override
public List<_ThreePid> getThreepids(_MatrixID userId) {
return doRequest(userId, p -> Optional.ofNullable(p.getThreepids()), profile -> {
List<_ThreePid> t = new ArrayList<>();
if (Objects.nonNull(profile.getThreepids())) {
t.addAll(profile.getThreepids());
}
return Optional.of(t);
}).orElseGet(Collections::emptyList);
}
@Override
public List<String> getRoles(_MatrixID userId) {
return doRequest(userId, p -> Optional.ofNullable(p.getRoles()), profile -> {
List<String> t = new ArrayList<>();
if (Objects.nonNull(profile.getRoles())) {
t.addAll(profile.getRoles());
}
return Optional.of(t);
}).orElseGet(Collections::emptyList);
}
}

75
src/main/java/io/kamax/mxisd/config/rest/RestBackendConfig.java
View File

@@ -30,6 +30,8 @@ import org.springframework.context.annotation.Configuration;
import javax.annotation.PostConstruct;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Objects;
import java.util.Optional;
@Configuration
@ConfigurationProperties("rest")
@@ -58,11 +60,44 @@ public class RestBackendConfig {
}
public static class ProfileEndpoints {
private String displayName;
private String threepids;
private String roles;
public String getDisplayName() {
return displayName;
}
public void setDisplayName(String displayName) {
this.displayName = displayName;
}
public String getThreepids() {
return threepids;
}
public void setThreepids(String threepids) {
this.threepids = threepids;
}
public String getRoles() {
return roles;
}
public void setRoles(String roles) {
this.roles = roles;
}
}
public static class Endpoints {
private String auth;
private String directory;
private IdentityEndpoints identity = new IdentityEndpoints();
private ProfileEndpoints profile;
public String getAuth() {
return auth;
@@ -88,6 +123,14 @@ public class RestBackendConfig {
this.identity = identity;
}
public Optional<ProfileEndpoints> getProfile() {
return Optional.ofNullable(profile);
}
public void setProfile(ProfileEndpoints profile) {
this.profile = profile;
}
}
private Logger log = LoggerFactory.getLogger(RestBackendConfig.class);
@@ -121,21 +164,21 @@ public class RestBackendConfig {
}
private String buildEndpointUrl(String endpoint) {
if (StringUtils.startsWith(endpoint, "/")) {
if (StringUtils.isBlank(getHost())) {
throw new ConfigurationException("rest.host");
}
try {
new URL(getHost());
} catch (MalformedURLException e) {
throw new ConfigurationException("rest.host", e.getMessage());
}
return getHost() + endpoint;
} else {
if (!StringUtils.startsWith(endpoint, "/")) {
return endpoint;
}
if (StringUtils.isBlank(getHost())) {
throw new ConfigurationException("rest.host");
}
try {
new URL(getHost());
} catch (MalformedURLException e) {
throw new ConfigurationException("rest.host", e.getMessage());
}
return getHost() + endpoint;
}
@PostConstruct
@@ -149,6 +192,12 @@ public class RestBackendConfig {
endpoints.identity.setSingle(buildEndpointUrl(endpoints.identity.getSingle()));
endpoints.identity.setBulk(buildEndpointUrl(endpoints.identity.getBulk()));
if (Objects.nonNull(endpoints.profile)) {
endpoints.profile.setDisplayName(buildEndpointUrl(endpoints.profile.getDisplayName()));
endpoints.profile.setThreepids(buildEndpointUrl(endpoints.profile.getThreepids()));
endpoints.profile.setRoles(buildEndpointUrl(endpoints.profile.getRoles()));
}
log.info("Host: {}", getHost());
log.info("Auth endpoint: {}", endpoints.getAuth());
log.info("Directory endpoint: {}", endpoints.getDirectory());

4
src/main/resources/application.yaml
View File

@@ -56,6 +56,10 @@ rest:
identity:
single: '/_mxisd/backend/api/v1/identity/lookup/single'
bulk: '/_mxisd/backend/api/v1/identity/lookup/bulk'
profile:
displayName: '/_mxisd/backend/api/v1/profile/displayName'
threepids: '/_mxisd/backend/api/v1/profile/threepids'
roles: '/_mxisd/backend/api/v1/profile/roles'
ldap:
enabled: false

146
src/test/java/io/kamax/mxisd/backend/rest/RestProfileProviderTest.java Normal file
View File

@@ -0,0 +1,146 @@
/*
* mxisd - Matrix Identity Server Daemon
* Copyright (C) 2018 Kamax Sarl
*
* https://www.kamax.io/
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
package io.kamax.mxisd.backend.rest;
import com.github.tomakehurst.wiremock.junit.WireMockRule;
import io.kamax.matrix.MatrixID;
import io.kamax.matrix._MatrixID;
import io.kamax.matrix.json.GsonUtil;
import io.kamax.mxisd.config.rest.RestBackendConfig;
import io.kamax.mxisd.exception.InternalServerError;
import io.kamax.mxisd.profile.JsonProfileRequest;
import io.kamax.mxisd.profile.JsonProfileResult;
import org.apache.http.entity.ContentType;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import java.util.Optional;
import static com.github.tomakehurst.wiremock.client.WireMock.*;
import static io.kamax.mxisd.config.rest.RestBackendConfig.ProfileEndpoints;
import static org.junit.Assert.*;
public class RestProfileProviderTest {
@Rule
public WireMockRule wireMockRule = new WireMockRule(65000);
private final String displayNameEndpoint = "/displayName";
private final _MatrixID userId = MatrixID.from("john", "matrix.localhost").valid();
private RestProfileProvider p;
@Before
public void before() {
ProfileEndpoints endpoints = new ProfileEndpoints();
endpoints.setDisplayName(displayNameEndpoint);
RestBackendConfig cfg = new RestBackendConfig();
cfg.setEnabled(true);
cfg.setHost("http://localhost:65000");
cfg.getEndpoints().setProfile(endpoints);
cfg.build();
p = new RestProfileProvider(cfg);
}
@Test
public void forNameFound() {
String value = "This is my display name";
JsonProfileResult r = new JsonProfileResult();
r.setDisplayName(value);
stubFor(post(urlEqualTo(displayNameEndpoint))
.willReturn(aResponse()
.withHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType())
.withBody(GsonUtil.get().toJson(GsonUtil.makeObj("profile", r)))
)
);
Optional<String> v = p.getDisplayName(userId);
verify(postRequestedFor(urlMatching(displayNameEndpoint))
.withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
.withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
);
assertTrue(v.isPresent());
assertEquals(value, v.get());
}
@Test
public void forNameNotFound() {
stubFor(post(urlEqualTo(displayNameEndpoint))
.willReturn(aResponse()
.withStatus(404)
)
);
Optional<String> v = p.getDisplayName(userId);
verify(postRequestedFor(urlMatching(displayNameEndpoint))
.withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
.withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
);
assertFalse(v.isPresent());
}
@Test
public void forNameEmptyBody() {
stubFor(post(urlEqualTo(displayNameEndpoint))
.willReturn(aResponse()
.withHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType())
)
);
Optional<String> v = p.getDisplayName(userId);
verify(postRequestedFor(urlMatching(displayNameEndpoint))
.withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
.withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
);
assertFalse(v.isPresent());
}
@Test(expected = InternalServerError.class)
public void forNameInvalidBody() {
stubFor(post(urlEqualTo(displayNameEndpoint))
.willReturn(aResponse()
.withHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType())
.withBody("This is not a valid JSON object")
)
);
try {
p.getDisplayName(userId);
} finally {
verify(postRequestedFor(urlMatching(displayNameEndpoint))
.withHeader("Content-Type", containing(ContentType.APPLICATION_JSON.getMimeType()))
.withRequestBody(equalTo(GsonUtil.get().toJson(new JsonProfileRequest(userId))))
);
}
}
}