116 lines
5.8 KiB
Markdown
116 lines
5.8 KiB
Markdown
# Admin Operations Center (AOC)
|
||
|
||
FastAPI microservice that ingests Microsoft Entra (Azure AD) and other admin audit logs into MongoDB, dedupes them, and exposes a UI/API to fetch, search, and review events.
|
||
|
||
## Components
|
||
- FastAPI app under `backend/` with routes to fetch audit logs and list stored events.
|
||
- MongoDB for persistence (provisioned via Docker Compose).
|
||
- Microsoft Graph client (client credentials) for retrieving directory audit events and Intune audit events.
|
||
- Office 365 Management Activity API client for Exchange/SharePoint/Teams admin audit logs.
|
||
- Frontend served from the backend for filtering/searching events and viewing raw entries.
|
||
- Optional OIDC bearer auth (Entra) to protect the API/UI and gate access by roles/groups.
|
||
|
||
## Prerequisites (macOS)
|
||
- Python 3.11
|
||
- Docker Desktop (for the quickest start) or a local MongoDB instance
|
||
- An Entra app registration with **Application** permission `AuditLog.Read.All` and admin consent granted
|
||
- Also required to fetch other sources:
|
||
- `https://manage.office.com/.default` (Audit API) with `ActivityFeed.Read`/`ActivityFeed.ReadDlp` (built into the app registration’s API permissions for Office 365 Management APIs)
|
||
- Intune audit: `DeviceManagementConfiguration.Read.All` (or broader) for `/deviceManagement/auditEvents`
|
||
- Optional API protection: configure `AUTH_ENABLED=true` and set `AUTH_TENANT_ID`/`AUTH_CLIENT_ID` (the audience) plus allowed roles/groups.
|
||
|
||
## Configuration
|
||
Create a `.env` file at the repo root (copy `.env.example`) and fill in your Microsoft Graph app credentials. The provided `MONGO_URI` works with the bundled MongoDB container; change it if you use a different Mongo instance.
|
||
|
||
```bash
|
||
cp .env.example .env
|
||
# edit .env to add TENANT_ID, CLIENT_ID, CLIENT_SECRET (and MONGO_URI if needed)
|
||
# optional: enable auth & periodic fetch
|
||
# AUTH_ENABLED=true
|
||
# AUTH_TENANT_ID=...
|
||
# AUTH_CLIENT_ID=...
|
||
# AUTH_ALLOWED_ROLES=Admins,SecurityOps
|
||
# ENABLE_PERIODIC_FETCH=true
|
||
# FETCH_INTERVAL_MINUTES=60
|
||
```
|
||
|
||
## Run with Docker Compose (recommended)
|
||
```bash
|
||
docker compose up --build
|
||
```
|
||
- API: http://localhost:8000
|
||
- Frontend: http://localhost:8000
|
||
- Mongo: localhost:27017 (root/example)
|
||
|
||
## Run locally without Docker
|
||
1) Start MongoDB (e.g. with Docker):
|
||
`docker run --rm -p 27017:27017 -e MONGO_INITDB_ROOT_USERNAME=root -e MONGO_INITDB_ROOT_PASSWORD=example mongo:7`
|
||
|
||
2) Prepare the backend environment:
|
||
```bash
|
||
cd backend
|
||
python3 -m venv .venv
|
||
source .venv/bin/activate
|
||
pip install -r requirements.txt
|
||
export $(cat ../.env | xargs) # or set env vars manually
|
||
uvicorn main:app --reload --host 0.0.0.0 --port 8000
|
||
```
|
||
|
||
## API
|
||
- `GET /api/fetch-audit-logs` — pulls the last 7 days by default (override with `?hours=N`, capped to 30 days) of:
|
||
- Entra directory audit logs (`/auditLogs/directoryAudits`)
|
||
- Exchange/SharePoint/Teams admin audits (via Office 365 Management Activity API)
|
||
- Intune audit logs (`/deviceManagement/auditEvents`)
|
||
Dedupes on a stable key (source id or timestamp/category/operation/target). Returns count and per-source warnings.
|
||
- `GET /api/events` — list stored events with filters:
|
||
- `service`, `actor`, `operation`, `result`, `start`, `end`, `search` (free text over raw/summary/actor/targets)
|
||
- Pagination: `page`, `page_size` (defaults 1, 50; max 500)
|
||
- `GET /api/filter-options` — best-effort distinct values for services, operations, results, actors (used by UI dropdowns).
|
||
|
||
Stored document shape (collection `micro_soc.events`):
|
||
```json
|
||
{
|
||
"id": "...", // original source id
|
||
"timestamp": "...", // activityDateTime
|
||
"service": "...", // category
|
||
"operation": "...", // activityDisplayName
|
||
"result": "...",
|
||
"actor_display": "...", // resolved user/app name
|
||
"target_displays": [ ... ],
|
||
"display_summary": "...",
|
||
"dedupe_key": "...", // used for upserts
|
||
"actor": { ... }, // initiatedBy
|
||
"targets": [ ... ], // targetResources
|
||
"raw": { ... }, // full source event
|
||
"raw_text": "..." // raw as string for text search
|
||
}
|
||
```
|
||
|
||
## Quick smoke tests
|
||
With the server running:
|
||
```bash
|
||
curl http://localhost:8000/api/events
|
||
curl http://localhost:8000/api/fetch-audit-logs
|
||
```
|
||
- Visit the UI at http://localhost:8000 to filter by user/service/action/result/time, search raw text, paginate, and view raw events.
|
||
|
||
## Maintenance (Dockerized)
|
||
Use the backend image so you don’t need a local venv:
|
||
```bash
|
||
# ensure Mongo + backend network are up
|
||
docker compose up -d mongo
|
||
# re-run enrichment/normalization on stored events (uses .env for Graph/Mongo)
|
||
docker compose run --rm backend python maintenance.py renormalize --limit 500
|
||
# deduplicate existing events (optional)
|
||
docker compose run --rm backend python maintenance.py dedupe
|
||
```
|
||
Omit `--limit` to process all events. You can also run commands inside a running backend container with `docker compose exec backend ...`.
|
||
|
||
## Notes / Troubleshooting
|
||
- Ensure `TENANT_ID`, `CLIENT_ID`, and `CLIENT_SECRET` match an app registration with `AuditLog.Read.All` (application) permission and admin consent.
|
||
- Additional permissions: Office 365 Management Activity (`ActivityFeed.Read`), and Intune audit (`DeviceManagementConfiguration.Read.All`).
|
||
- Auth: if `AUTH_ENABLED=true`, issued tokens must be from `AUTH_TENANT_ID`, audience = `AUTH_CLIENT_ID`; access is granted if roles or groups overlap `AUTH_ALLOWED_ROLES`/`AUTH_ALLOWED_GROUPS` (if set).
|
||
- Backfill limits: Management Activity API typically exposes ~7 days of history via API (longer if your tenant has extended/Advanced Audit retention). Directory/Intune audit retention follows your tenant policy (commonly 30–90 days, longer with Advanced Audit).
|
||
- If you change Mongo credentials/ports, update `MONGO_URI` in `.env` (Docker Compose passes it through to the backend).
|
||
- The service uses the `micro_soc` database and `events` collection by default; adjust in `backend/config.py` if needed.
|