docs: update AGENTS.md, README.md, DEPLOY.md, ROADMAP.md for v1.7.14 security features

README.md

@@ -11,13 +11,14 @@ FastAPI microservice that ingests Microsoft Entra (Azure AD) and other admin aud
 - Optional OIDC bearer auth (Entra) to protect the API/UI and gate access by roles/groups.
 - Natural language query (`/api/ask`) powered by LLM (OpenAI, Azure OpenAI, or any compatible API).
 - MCP server for Claude Desktop / Cursor integration.
+- Optional Azure Key Vault integration for secrets storage.
 
 ## Prerequisites (macOS)
 - Python 3.11
 - Docker Desktop (for the quickest start) or a local MongoDB instance
 - An Entra app registration with **Application** permission `AuditLog.Read.All` and admin consent granted
   - Also required to fetch other sources:
-    - `https://manage.office.com/.default` (Audit API) with `ActivityFeed.Read`/`ActivityFeed.ReadDlp` (built into the app registration’s API permissions for Office 365 Management APIs)
+    - `https://manage.office.com/.default` (Audit API) with `ActivityFeed.Read`/`ActivityFeed.ReadDlp` (built into the app registration's API permissions for Office 365 Management APIs)
     - Intune audit: `DeviceManagementConfiguration.Read.All` (or broader) for `/deviceManagement/auditEvents`
   - Optional API protection: configure `AUTH_ENABLED=true` and set `AUTH_TENANT_ID`/`AUTH_CLIENT_ID` (the audience) plus allowed roles/groups.
 
@@ -49,8 +50,43 @@ cp .env.example .env
 # LLM_BASE_URL=https://api.openai.com/v1
 # LLM_MODEL=gpt-4o-mini
 # LLM_TIMEOUT_SECONDS=30
+# LLM_ALLOWED_DOMAINS=api.openai.com,*.openai.azure.com
+
+# Optional: SIEM forwarding
+# SIEM_ENABLED=true
+# SIEM_WEBHOOK_URL=https://your-siem.com/webhook
+# SIEM_ALLOWED_DOMAINS=your-siem.com
+
+# Optional: Azure Key Vault for secrets storage
+# AZURE_KEY_VAULT_NAME=your-keyvault-name
 ```
 
+### Using Azure Key Vault for secrets
+Instead of storing `CLIENT_SECRET`, `LLM_API_KEY`, `MONGO_URI`, and `WEBHOOK_CLIENT_SECRET` in `.env`, you can store them in Azure Key Vault:
+
+1. Create a Key Vault and add secrets with these names:
+   - `aoc-client-secret` → your Graph app `CLIENT_SECRET`
+   - `aoc-llm-api-key` → your `LLM_API_KEY`
+   - `aoc-mongo-uri` → your `MONGO_URI`
+   - `aoc-webhook-client-secret` → your `WEBHOOK_CLIENT_SECRET`
+2. Uncomment `azure-identity` and `azure-keyvault-secrets` in `backend/requirements.txt`
+3. Set `AZURE_KEY_VAULT_NAME=your-keyvault-name` in `.env`
+4. Ensure the container has Azure identity credentials (managed identity, service principal, or Azure CLI auth)
+
+## Security Hardening Checklist
+
+Before deploying to production:
+
+- [ ] Set `AUTH_ENABLED=true` and configure `AUTH_ALLOWED_ROLES` or `AUTH_ALLOWED_GROUPS` to restrict access
+- [ ] Set explicit `CORS_ORIGINS` (do not use `*` in production with auth enabled)
+- [ ] Set `DOCS_ENABLED=false` (default) to hide OpenAPI docs
+- [ ] Configure `WEBHOOK_CLIENT_SECRET` to validate Graph webhook notifications
+- [ ] Set `LLM_ALLOWED_DOMAINS` if using AI features to prevent data exfiltration
+- [ ] Set `SIEM_ALLOWED_DOMAINS` if using SIEM forwarding
+- [ ] Review `METRICS_ALLOWED_IPS` — defaults to private networks only
+- [ ] Consider Azure Key Vault instead of `.env` for secrets
+- [ ] Review the threat model: `THREAT_MODEL_v1.7.13.md`
+
 ## Run with Docker Compose (recommended)
 ```bash
 docker compose up --build
@@ -76,7 +112,7 @@ uvicorn main:app --reload --host 0.0.0.0 --port 8000
 
 ## API
 - `GET /health` — health check with MongoDB connectivity status.
-- `GET /metrics` — Prometheus metrics for request latency, fetch volume, and errors.
+- `GET /metrics` — Prometheus metrics for request latency, fetch volume, and errors (IP-restricted).
 - `GET /api/version` — running version (baked into the Docker image at build time).
 - `GET /api/fetch-audit-logs` — pulls the last 7 days by default (override with `?hours=N`, capped to 30 days) of:
   - Entra directory audit logs (`/auditLogs/directoryAudits`)
@@ -171,7 +207,7 @@ curl http://localhost:8000/api/fetch-audit-logs
 - Visit the UI at http://localhost:8000 to filter by user/service/action/result/time, search raw text, paginate, and view raw events.
 
 ## Maintenance (Dockerized)
-Use the backend image so you don’t need a local venv:
+Use the backend image so you don't need a local venv:
 ```bash
 # ensure Mongo + backend network are up
 docker compose up -d mongo
@@ -182,10 +218,15 @@ docker compose run --rm backend python maintenance.py dedupe
 ```
 Omit `--limit` to process all events. You can also run commands inside a running backend container with `docker compose exec backend ...`.
 
+## Security Documentation
+- `PEN_TEST_REPORT_v1.7.11.md` — Penetration test findings and remediation
+- `THREAT_MODEL_v1.7.13.md` — Comprehensive threat model covering Entra application abuse, token handling, data exfiltration vectors
+
 ## Notes / Troubleshooting
 - Ensure `TENANT_ID`, `CLIENT_ID`, and `CLIENT_SECRET` match an app registration with `AuditLog.Read.All` (application) permission and admin consent.
 - Additional permissions: Office 365 Management Activity (`ActivityFeed.Read`), and Intune audit (`DeviceManagementConfiguration.Read.All`).
-- Auth: if `AUTH_ENABLED=true`, issued tokens must be from `AUTH_TENANT_ID`, audience = `AUTH_CLIENT_ID`; access is granted if roles or groups overlap `AUTH_ALLOWED_ROLES`/`AUTH_ALLOWED_GROUPS` (if set).
+- Auth: if `AUTH_ENABLED=true`, issued tokens must be from `AUTH_TENANT_ID`, audience = `AUTH_CLIENT_ID`; access is granted if roles or groups overlap `AUTH_ALLOWED_ROLES`/`AUTH_ALLOWED_GROUPS` (if set). A startup warning is logged if auth is enabled but no roles/groups are configured.
 - Backfill limits: Management Activity API typically exposes ~7 days of history via API (longer if your tenant has extended/Advanced Audit retention). Directory/Intune audit retention follows your tenant policy (commonly 30–90 days, longer with Advanced Audit).
 - If you change Mongo credentials/ports, update `MONGO_URI` in `.env` (Docker Compose passes it through to the backend).
 - The service uses the `micro_soc` database and `events` collection by default; adjust in `backend/config.py` if needed.
+- If using Azure Key Vault, ensure the runtime identity (managed identity, service principal, or local Azure CLI) has `Get` permission on secrets.