Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 11fd87411d | |||
| 6a80bf4eb9 | |||
| 5e02f5a402 | |||
| 0c3e5ec57b | |||
| a255be93fe | |||
| cfe9397cc5 | |||
| cf0283b20b | |||
| 28542f7b80 |
@@ -42,6 +42,6 @@ ALERTS_ENABLED=false
|
|||||||
LLM_API_KEY=
|
LLM_API_KEY=
|
||||||
LLM_BASE_URL=https://api.openai.com/v1
|
LLM_BASE_URL=https://api.openai.com/v1
|
||||||
LLM_MODEL=gpt-4o-mini
|
LLM_MODEL=gpt-4o-mini
|
||||||
LLM_MAX_EVENTS=50
|
LLM_MAX_EVENTS=200
|
||||||
LLM_TIMEOUT_SECONDS=30
|
LLM_TIMEOUT_SECONDS=30
|
||||||
LLM_API_VERSION=
|
LLM_API_VERSION=
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ jobs:
|
|||||||
run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login git.cqre.net -u ${{ github.actor }} --password-stdin 2>&1 | grep -v "WARNING! Your credentials are stored unencrypted"
|
run: echo "${{ secrets.REGISTRY_TOKEN }}" | docker login git.cqre.net -u ${{ github.actor }} --password-stdin 2>&1 | grep -v "WARNING! Your credentials are stored unencrypted"
|
||||||
|
|
||||||
- name: Build Docker image
|
- name: Build Docker image
|
||||||
run: docker build ./backend --tag git.cqre.net/cqrenet/aoc-backend:${{ gitea.ref_name }}
|
run: docker build ./backend --build-arg VERSION=${{ gitea.ref_name }} --tag git.cqre.net/cqrenet/aoc-backend:${{ gitea.ref_name }}
|
||||||
|
|
||||||
- name: Push Docker image
|
- name: Push Docker image
|
||||||
run: docker push git.cqre.net/cqrenet/aoc-backend:${{ gitea.ref_name }}
|
run: docker push git.cqre.net/cqrenet/aoc-backend:${{ gitea.ref_name }}
|
||||||
|
|||||||
56
RELEASE_NOTES_v1.1.0.md
Normal file
56
RELEASE_NOTES_v1.1.0.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
# AOC v1.1.0 Release Notes
|
||||||
|
|
||||||
|
**Release date:** 2026-04-20
|
||||||
|
|
||||||
|
## What's new
|
||||||
|
|
||||||
|
### Natural language query (`/api/ask`)
|
||||||
|
Ask questions in plain English and get AI-generated answers backed by your audit logs.
|
||||||
|
|
||||||
|
- **Regex-based parsing** extracts time ranges (`last 3 days`, `yesterday`, `today`) and entities (`device ABC123`, `user bob@example.com`) without calling an LLM — fast and deterministic.
|
||||||
|
- **AI narrative summarisation** via any OpenAI-compatible API (OpenAI, Azure OpenAI, MS Foundry, Ollama). The LLM reads the matching events and writes a concise story for non-expert admins.
|
||||||
|
- **Graceful fallback** when no LLM is configured — returns a structured bullet list instead of a narrative.
|
||||||
|
- **Cited evidence** — every answer includes the raw events that back it up, so admins can verify claims.
|
||||||
|
|
||||||
|
### Azure OpenAI / MS Foundry support
|
||||||
|
- Automatic `api-key` header detection for Azure endpoints.
|
||||||
|
- `LLM_API_VERSION` config for Azure `api-version` query parameters.
|
||||||
|
- `max_completion_tokens` support for newer model deployments.
|
||||||
|
|
||||||
|
### Production hardening
|
||||||
|
- **Dockerfile:** runs as non-root user, uses Gunicorn + Uvicorn workers.
|
||||||
|
- **docker-compose.prod.yml:** MongoDB is internal-only (no host port exposure), health checks on all services, nginx reverse proxy with security headers.
|
||||||
|
- **nginx config:** gzip, security headers (`X-Frame-Options`, `X-Content-Type-Options`), ready for TLS.
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
- New **"Ask a question"** panel at the top of the page.
|
||||||
|
- Markdown rendering for LLM answers (bold, italic, code).
|
||||||
|
- Orange warning banner when LLM is not configured or fails.
|
||||||
|
|
||||||
|
### Tests
|
||||||
|
- 29 new tests covering ask parsing, query building, and endpoint behaviour.
|
||||||
|
- 62 tests total, all passing.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Add to your `.env`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Required for AI narrative summarisation
|
||||||
|
LLM_API_KEY=your-key
|
||||||
|
LLM_BASE_URL=https://api.openai.com/v1
|
||||||
|
LLM_MODEL=gpt-4o-mini
|
||||||
|
LLM_MAX_EVENTS=50
|
||||||
|
LLM_TIMEOUT_SECONDS=30
|
||||||
|
LLM_API_VERSION= # set for Azure OpenAI, e.g. 2024-12-01-preview
|
||||||
|
```
|
||||||
|
|
||||||
|
## Upgrade notes
|
||||||
|
|
||||||
|
No breaking changes. Existing `/api/events`, filters, pagination, tags, and comments work unchanged.
|
||||||
|
|
||||||
|
## Docker image
|
||||||
|
|
||||||
|
```
|
||||||
|
git.cqre.net/cqrenet/aoc-backend:v1.1.0
|
||||||
|
```
|
||||||
78
RELEASE_NOTES_v1.2.5.md
Normal file
78
RELEASE_NOTES_v1.2.5.md
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
# AOC v1.2.5 Release Notes
|
||||||
|
|
||||||
|
**Release date:** 2026-04-20
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What's new
|
||||||
|
|
||||||
|
### Natural language query (`/api/ask`)
|
||||||
|
Ask questions in plain English and get AI-generated answers backed by your audit logs.
|
||||||
|
|
||||||
|
- **Regex-based parsing** extracts time ranges (`last 3 days`, `yesterday`, `today`) and entities (`device ABC123`, `user bob@example.com`) without calling an LLM.
|
||||||
|
- **AI narrative summarisation** via any OpenAI-compatible API (OpenAI, Azure OpenAI, MS Foundry, Ollama).
|
||||||
|
- **Graceful fallback** when no LLM is configured — returns a structured bullet list with a clear error banner.
|
||||||
|
- **Cited evidence** — every answer includes the raw events that back it up.
|
||||||
|
|
||||||
|
### Filter-aware queries
|
||||||
|
The ask endpoint now respects the filter panel. When you set **Service = Exchange**, **Result = failure** and ask *"What happened to device X?"*, the LLM only sees failed Exchange events for that device.
|
||||||
|
|
||||||
|
### Scales to thousands of events
|
||||||
|
For large result sets (>50 events), the LLM receives an **aggregated overview** instead of a raw dump:
|
||||||
|
- Counts by service, action, result, and actor
|
||||||
|
- Failure highlights
|
||||||
|
- The 50 most recent raw events as samples
|
||||||
|
|
||||||
|
This keeps token usage low while preserving accuracy.
|
||||||
|
|
||||||
|
### Azure OpenAI / MS Foundry support
|
||||||
|
- Automatic `api-key` header detection for Azure endpoints.
|
||||||
|
- `LLM_API_VERSION` config for Azure `api-version` query parameters.
|
||||||
|
- `max_completion_tokens` support for newer model deployments.
|
||||||
|
|
||||||
|
### Version display
|
||||||
|
- `GET /api/version` endpoint reads the `VERSION` file.
|
||||||
|
- Frontend shows a version badge in the header (e.g., **1.2.5**).
|
||||||
|
|
||||||
|
### Production hardening (from v1.1.0)
|
||||||
|
- Dockerfile runs as non-root user with Gunicorn + Uvicorn workers.
|
||||||
|
- `docker-compose.prod.yml` with internal-only MongoDB, health checks, and nginx reverse proxy.
|
||||||
|
- Security headers (`X-Frame-Options`, `X-Content-Type-Options`, etc.).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Add to your `.env`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Required for AI narrative summarisation
|
||||||
|
LLM_API_KEY=your-key
|
||||||
|
LLM_BASE_URL=https://api.openai.com/v1
|
||||||
|
LLM_MODEL=gpt-4o-mini
|
||||||
|
LLM_MAX_EVENTS=200
|
||||||
|
LLM_TIMEOUT_SECONDS=30
|
||||||
|
LLM_API_VERSION= # set for Azure OpenAI, e.g. 2024-12-01-preview
|
||||||
|
```
|
||||||
|
|
||||||
|
For Azure OpenAI / MS Foundry:
|
||||||
|
```bash
|
||||||
|
LLM_BASE_URL=https://your-resource.openai.azure.com/openai/deployments/your-deployment
|
||||||
|
LLM_API_KEY=your-azure-key
|
||||||
|
LLM_API_VERSION=2024-12-01-preview
|
||||||
|
LLM_MODEL=your-deployment-name
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Upgrade notes
|
||||||
|
|
||||||
|
No breaking changes. Existing `/api/events`, filters, pagination, tags, and comments work unchanged.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker image
|
||||||
|
|
||||||
|
```
|
||||||
|
git.cqre.net/cqrenet/aoc-backend:v1.2.5
|
||||||
|
```
|
||||||
@@ -1,5 +1,9 @@
|
|||||||
FROM python:3.11-slim
|
FROM python:3.11-slim
|
||||||
|
|
||||||
|
# Bake the version into the image at build time
|
||||||
|
ARG VERSION=unknown
|
||||||
|
ENV VERSION=${VERSION}
|
||||||
|
|
||||||
# Security: run as non-root
|
# Security: run as non-root
|
||||||
RUN groupadd -r aoc && useradd -r -g aoc aoc
|
RUN groupadd -r aoc && useradd -r -g aoc aoc
|
||||||
|
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ class Settings(BaseSettings):
|
|||||||
LLM_API_KEY: str = ""
|
LLM_API_KEY: str = ""
|
||||||
LLM_BASE_URL: str = "https://api.openai.com/v1"
|
LLM_BASE_URL: str = "https://api.openai.com/v1"
|
||||||
LLM_MODEL: str = "gpt-4o-mini"
|
LLM_MODEL: str = "gpt-4o-mini"
|
||||||
LLM_MAX_EVENTS: int = 50
|
LLM_MAX_EVENTS: int = 200
|
||||||
LLM_TIMEOUT_SECONDS: int = 30
|
LLM_TIMEOUT_SECONDS: int = 30
|
||||||
LLM_API_VERSION: str = "" # e.g. 2025-01-01-preview for Azure OpenAI
|
LLM_API_VERSION: str = "" # e.g. 2025-01-01-preview for Azure OpenAI
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,7 @@
|
|||||||
<div class="page" x-data="aocApp()" x-init="initApp()">
|
<div class="page" x-data="aocApp()" x-init="initApp()">
|
||||||
<header class="hero">
|
<header class="hero">
|
||||||
<div>
|
<div>
|
||||||
<p class="eyebrow">Admin Operations Center</p>
|
<p class="eyebrow">Admin Operations Center <span class="version-badge" x-text="appVersion"></span></p>
|
||||||
<h1>Directory Audit Explorer</h1>
|
<h1>Directory Audit Explorer</h1>
|
||||||
<p class="lede">Filter Microsoft Entra audit events by user, app, time, action, and action type.</p>
|
<p class="lede">Filter Microsoft Entra audit events by user, app, time, action, and action type.</p>
|
||||||
</div>
|
</div>
|
||||||
@@ -50,6 +50,9 @@
|
|||||||
/>
|
/>
|
||||||
<button type="submit" :disabled="askLoading" x-text="askLoading ? 'Thinking…' : 'Ask'">Ask</button>
|
<button type="submit" :disabled="askLoading" x-text="askLoading ? 'Thinking…' : 'Ask'">Ask</button>
|
||||||
</div>
|
</div>
|
||||||
|
<div x-show="hasActiveFilters()" class="ask-filter-hint">
|
||||||
|
<small>Respecting active filters: <span x-text="activeFilterSummary()"></span></small>
|
||||||
|
</div>
|
||||||
</form>
|
</form>
|
||||||
<template x-if="askAnswer">
|
<template x-if="askAnswer">
|
||||||
<div class="ask-result">
|
<div class="ask-result">
|
||||||
@@ -240,6 +243,7 @@
|
|||||||
actor: '', selectedServices: [], search: '', operation: '', result: '', start: '', end: '', limit: 100, includeTags: '', excludeTags: '',
|
actor: '', selectedServices: [], search: '', operation: '', result: '', start: '', end: '', limit: 100, includeTags: '', excludeTags: '',
|
||||||
},
|
},
|
||||||
options: { actors: [], services: [], operations: [], results: [] },
|
options: { actors: [], services: [], operations: [], results: [] },
|
||||||
|
appVersion: '',
|
||||||
askQuestionText: '',
|
askQuestionText: '',
|
||||||
askLoading: false,
|
askLoading: false,
|
||||||
askAnswer: '',
|
askAnswer: '',
|
||||||
@@ -249,6 +253,7 @@
|
|||||||
askLlmError: '',
|
askLlmError: '',
|
||||||
|
|
||||||
async initApp() {
|
async initApp() {
|
||||||
|
await this.loadVersion();
|
||||||
await this.initAuth();
|
await this.initAuth();
|
||||||
if (!this.authConfig?.auth_enabled || this.accessToken) {
|
if (!this.authConfig?.auth_enabled || this.accessToken) {
|
||||||
await this.loadFilterOptions();
|
await this.loadFilterOptions();
|
||||||
@@ -257,6 +262,16 @@
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
|
async loadVersion() {
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/version');
|
||||||
|
if (res.ok) {
|
||||||
|
const body = await res.json();
|
||||||
|
this.appVersion = body.version || '';
|
||||||
|
}
|
||||||
|
} catch {}
|
||||||
|
},
|
||||||
|
|
||||||
authHeader() {
|
authHeader() {
|
||||||
return this.accessToken ? { Authorization: `Bearer ${this.accessToken}` } : {};
|
return this.accessToken ? { Authorization: `Bearer ${this.accessToken}` } : {};
|
||||||
},
|
},
|
||||||
@@ -491,11 +506,29 @@
|
|||||||
this.askAnswer = '';
|
this.askAnswer = '';
|
||||||
this.askAnswerHtml = '';
|
this.askAnswerHtml = '';
|
||||||
this.askEvents = [];
|
this.askEvents = [];
|
||||||
|
this.askLlmError = '';
|
||||||
|
|
||||||
|
const payload = { question: q };
|
||||||
|
if (this.filters.selectedServices && this.filters.selectedServices.length) {
|
||||||
|
payload.services = this.filters.selectedServices;
|
||||||
|
}
|
||||||
|
if (this.filters.actor) payload.actor = this.filters.actor;
|
||||||
|
if (this.filters.operation) payload.operation = this.filters.operation;
|
||||||
|
if (this.filters.result) payload.result = this.filters.result;
|
||||||
|
if (this.filters.start) payload.start = new Date(this.filters.start).toISOString();
|
||||||
|
if (this.filters.end) payload.end = new Date(this.filters.end).toISOString();
|
||||||
|
if (this.filters.includeTags) {
|
||||||
|
payload.include_tags = this.filters.includeTags.split(/[,;]+/).map(t => t.trim()).filter(Boolean);
|
||||||
|
}
|
||||||
|
if (this.filters.excludeTags) {
|
||||||
|
payload.exclude_tags = this.filters.excludeTags.split(/[,;]+/).map(t => t.trim()).filter(Boolean);
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const res = await fetch('/api/ask', {
|
const res = await fetch('/api/ask', {
|
||||||
method: 'POST',
|
method: 'POST',
|
||||||
headers: { 'Content-Type': 'application/json', ...this.authHeader() },
|
headers: { 'Content-Type': 'application/json', ...this.authHeader() },
|
||||||
body: JSON.stringify({ question: q }),
|
body: JSON.stringify(payload),
|
||||||
});
|
});
|
||||||
if (!res.ok) throw new Error(await res.text());
|
if (!res.ok) throw new Error(await res.text());
|
||||||
const body = await res.json();
|
const body = await res.json();
|
||||||
@@ -532,6 +565,27 @@
|
|||||||
.replace(/\n/g, '<br>');
|
.replace(/\n/g, '<br>');
|
||||||
},
|
},
|
||||||
|
|
||||||
|
hasActiveFilters() {
|
||||||
|
return this.filters.actor || this.filters.operation || this.filters.result ||
|
||||||
|
this.filters.start || this.filters.end || this.filters.includeTags ||
|
||||||
|
this.filters.excludeTags ||
|
||||||
|
(this.filters.selectedServices && this.filters.selectedServices.length &&
|
||||||
|
this.filters.selectedServices.length < this.options.services.length);
|
||||||
|
},
|
||||||
|
|
||||||
|
activeFilterSummary() {
|
||||||
|
const parts = [];
|
||||||
|
if (this.filters.actor) parts.push('actor');
|
||||||
|
if (this.filters.operation) parts.push('action');
|
||||||
|
if (this.filters.result) parts.push('result');
|
||||||
|
if (this.filters.start || this.filters.end) parts.push('time');
|
||||||
|
if (this.filters.includeTags || this.filters.excludeTags) parts.push('tags');
|
||||||
|
const svcCount = this.filters.selectedServices?.length || 0;
|
||||||
|
const allCount = this.options.services?.length || 0;
|
||||||
|
if (svcCount && svcCount < allCount) parts.push(`${svcCount} service${svcCount === 1 ? '' : 's'}`);
|
||||||
|
return parts.join(', ') || 'none';
|
||||||
|
},
|
||||||
|
|
||||||
async bulkTagMatching() {
|
async bulkTagMatching() {
|
||||||
const tag = prompt('Enter tag to apply to all matching events:');
|
const tag = prompt('Enter tag to apply to all matching events:');
|
||||||
if (!tag || !tag.trim()) return;
|
if (!tag || !tag.trim()) return;
|
||||||
|
|||||||
@@ -428,6 +428,25 @@ input {
|
|||||||
margin-bottom: 10px;
|
margin-bottom: 10px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.ask-filter-hint {
|
||||||
|
margin-top: 6px;
|
||||||
|
color: var(--muted);
|
||||||
|
}
|
||||||
|
|
||||||
|
.version-badge {
|
||||||
|
display: inline-block;
|
||||||
|
margin-left: 8px;
|
||||||
|
padding: 2px 8px;
|
||||||
|
border-radius: 999px;
|
||||||
|
background: rgba(125, 211, 252, 0.15);
|
||||||
|
border: 1px solid rgba(125, 211, 252, 0.3);
|
||||||
|
color: var(--accent-strong);
|
||||||
|
font-size: 11px;
|
||||||
|
font-weight: 600;
|
||||||
|
letter-spacing: 0.05em;
|
||||||
|
vertical-align: middle;
|
||||||
|
}
|
||||||
|
|
||||||
.ask-events {
|
.ask-events {
|
||||||
margin-bottom: 14px;
|
margin-bottom: 14px;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -134,6 +134,13 @@ async def metrics():
|
|||||||
return Response(content=prometheus_metrics(), media_type="text/plain")
|
return Response(content=prometheus_metrics(), media_type="text/plain")
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/api/version")
|
||||||
|
async def version():
|
||||||
|
import os
|
||||||
|
|
||||||
|
return {"version": os.environ.get("VERSION", "unknown")}
|
||||||
|
|
||||||
|
|
||||||
frontend_dir = Path(__file__).parent / "frontend"
|
frontend_dir = Path(__file__).parent / "frontend"
|
||||||
app.mount("/", StaticFiles(directory=frontend_dir, html=True), name="frontend")
|
app.mount("/", StaticFiles(directory=frontend_dir, html=True), name="frontend")
|
||||||
|
|
||||||
|
|||||||
@@ -74,6 +74,14 @@ class AlertRuleResponse(BaseModel):
|
|||||||
|
|
||||||
class AskRequest(BaseModel):
|
class AskRequest(BaseModel):
|
||||||
question: str
|
question: str
|
||||||
|
services: list[str] | None = None
|
||||||
|
actor: str | None = None
|
||||||
|
operation: str | None = None
|
||||||
|
result: str | None = None
|
||||||
|
start: str | None = None
|
||||||
|
end: str | None = None
|
||||||
|
include_tags: list[str] | None = None
|
||||||
|
exclude_tags: list[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
class AskEventRef(BaseModel):
|
class AskEventRef(BaseModel):
|
||||||
|
|||||||
@@ -104,7 +104,17 @@ def _extract_entity(question: str) -> str | None:
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def _build_event_query(entity: str | None, start: str | None, end: str | None) -> dict:
|
def _build_event_query(
|
||||||
|
entity: str | None,
|
||||||
|
start: str | None,
|
||||||
|
end: str | None,
|
||||||
|
services: list[str] | None = None,
|
||||||
|
actor: str | None = None,
|
||||||
|
operation: str | None = None,
|
||||||
|
result: str | None = None,
|
||||||
|
include_tags: list[str] | None = None,
|
||||||
|
exclude_tags: list[str] | None = None,
|
||||||
|
) -> dict:
|
||||||
filters = []
|
filters = []
|
||||||
|
|
||||||
if start or end:
|
if start or end:
|
||||||
@@ -128,6 +138,28 @@ def _build_event_query(entity: str | None, start: str | None, end: str | None) -
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if services:
|
||||||
|
filters.append({"service": {"$in": services}})
|
||||||
|
if actor:
|
||||||
|
actor_safe = re.escape(actor)
|
||||||
|
filters.append(
|
||||||
|
{
|
||||||
|
"$or": [
|
||||||
|
{"actor_display": {"$regex": actor_safe, "$options": "i"}},
|
||||||
|
{"actor_upn": {"$regex": actor_safe, "$options": "i"}},
|
||||||
|
{"actor.user.userPrincipalName": {"$regex": actor_safe, "$options": "i"}},
|
||||||
|
]
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if operation:
|
||||||
|
filters.append({"operation": {"$regex": re.escape(operation), "$options": "i"}})
|
||||||
|
if result:
|
||||||
|
filters.append({"result": {"$regex": re.escape(result), "$options": "i"}})
|
||||||
|
if include_tags:
|
||||||
|
filters.append({"tags": {"$all": include_tags}})
|
||||||
|
if exclude_tags:
|
||||||
|
filters.append({"tags": {"$not": {"$all": exclude_tags}}})
|
||||||
|
|
||||||
return {"$and": filters} if filters else {}
|
return {"$and": filters} if filters else {}
|
||||||
|
|
||||||
|
|
||||||
@@ -136,22 +168,76 @@ def _build_event_query(entity: str | None, start: str | None, end: str | None) -
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
_SYSTEM_PROMPT = """You are an IT operations assistant. An administrator has asked a question about audit logs.
|
_SYSTEM_PROMPT = """You are an IT operations assistant. An administrator has asked a question about audit logs.
|
||||||
Your job is to read the list of audit events below and write a concise, plain-language answer.
|
Your job is to read the data below and write a concise, plain-language answer.
|
||||||
|
|
||||||
|
The input may be either:
|
||||||
|
- A small list of individual audit events (numbered Event #1, #2, etc.), or
|
||||||
|
- An aggregated overview with counts by service, action, result, and actor, plus sample events.
|
||||||
|
|
||||||
Rules:
|
Rules:
|
||||||
- Assume the reader is a non-expert admin.
|
- Assume the reader is a non-expert admin.
|
||||||
- Group related events together and tell a coherent story.
|
- For aggregated overviews: summarise the scale, top patterns, and highlight anomalies or failures.
|
||||||
|
- For small event lists: group related events together and tell a coherent story.
|
||||||
- Highlight anything unusual, failed actions, or privilege escalations.
|
- Highlight anything unusual, failed actions, or privilege escalations.
|
||||||
- Reference specific event numbers (e.g., "Event #3") when making claims so the user can verify.
|
- Reference specific event numbers (e.g., "Event #3") when making claims so the user can verify.
|
||||||
|
- If the data is an aggregated subset of a larger result set, acknowledge the scale (e.g., "847 events occurred — the top pattern was...").
|
||||||
- If there are no events, say so clearly.
|
- If there are no events, say so clearly.
|
||||||
- Keep the answer under 300 words.
|
- Keep the answer under 300 words.
|
||||||
- Do not invent events that are not in the list.
|
- Do not invent events or patterns that are not supported by the data.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
def _format_events_for_llm(events: list[dict]) -> str:
|
def _aggregate_counts(events: list[dict]) -> dict:
|
||||||
|
"""Build lightweight aggregation tables for large result sets."""
|
||||||
|
from collections import Counter
|
||||||
|
|
||||||
|
svc_counts = Counter(e.get("service") or "Unknown" for e in events)
|
||||||
|
op_counts = Counter(e.get("operation") or "Unknown" for e in events)
|
||||||
|
result_counts = Counter(e.get("result") or "Unknown" for e in events)
|
||||||
|
actor_counts = Counter(e.get("actor_display") or "Unknown" for e in events)
|
||||||
|
return {
|
||||||
|
"services": svc_counts.most_common(10),
|
||||||
|
"operations": op_counts.most_common(10),
|
||||||
|
"results": result_counts.most_common(5),
|
||||||
|
"actors": actor_counts.most_common(10),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _format_events_for_llm(events: list[dict], total: int | None = None) -> str:
|
||||||
lines = []
|
lines = []
|
||||||
for i, e in enumerate(events, 1):
|
|
||||||
|
# If we have a large result set, send aggregation + samples instead of raw dump
|
||||||
|
if total is not None and total > len(events) and len(events) >= 50:
|
||||||
|
lines.append(f"Result set overview: {total} total events (showing the {len(events)} most recent).\n")
|
||||||
|
agg = _aggregate_counts(events)
|
||||||
|
lines.append("Breakdown by service:")
|
||||||
|
for svc, cnt in agg["services"]:
|
||||||
|
lines.append(f" {svc}: {cnt}")
|
||||||
|
lines.append("\nBreakdown by action:")
|
||||||
|
for op, cnt in agg["operations"]:
|
||||||
|
lines.append(f" {op}: {cnt}")
|
||||||
|
lines.append("\nBreakdown by result:")
|
||||||
|
for res, cnt in agg["results"]:
|
||||||
|
lines.append(f" {res}: {cnt}")
|
||||||
|
lines.append("\nTop actors:")
|
||||||
|
for actor, cnt in agg["actors"]:
|
||||||
|
lines.append(f" {actor}: {cnt}")
|
||||||
|
# Include failures and a few recent samples
|
||||||
|
failures = [e for e in events if str(e.get("result") or "").lower() in ("failure", "failed")]
|
||||||
|
if failures:
|
||||||
|
lines.append(f"\nFailures ({len(failures)}):")
|
||||||
|
for e in failures[:10]:
|
||||||
|
ts = e.get("timestamp", "?")[:16].replace("T", " ")
|
||||||
|
op = e.get("operation", "unknown")
|
||||||
|
actor = e.get("actor_display", "unknown")
|
||||||
|
lines.append(f" {ts} — {op} by {actor}")
|
||||||
|
lines.append("\nMost recent sample events:")
|
||||||
|
else:
|
||||||
|
if total is not None and total > len(events):
|
||||||
|
lines.append(f"Showing {len(events)} of {total} total matching events (most recent first):\n")
|
||||||
|
|
||||||
|
# Always include the first N raw events as detail (up to 50)
|
||||||
|
for i, e in enumerate(events[:50], 1):
|
||||||
ts = e.get("timestamp") or "unknown time"
|
ts = e.get("timestamp") or "unknown time"
|
||||||
op = e.get("operation") or "unknown action"
|
op = e.get("operation") or "unknown action"
|
||||||
actor = e.get("actor_display") or "unknown actor"
|
actor = e.get("actor_display") or "unknown actor"
|
||||||
@@ -181,11 +267,11 @@ def _build_chat_url(base_url: str, api_version: str) -> str:
|
|||||||
return url
|
return url
|
||||||
|
|
||||||
|
|
||||||
async def _call_llm(question: str, events: list[dict]) -> str:
|
async def _call_llm(question: str, events: list[dict], total: int | None = None) -> str:
|
||||||
if not LLM_API_KEY:
|
if not LLM_API_KEY:
|
||||||
raise RuntimeError("LLM_API_KEY not configured")
|
raise RuntimeError("LLM_API_KEY not configured")
|
||||||
|
|
||||||
context = _format_events_for_llm(events)
|
context = _format_events_for_llm(events, total=total)
|
||||||
messages = [
|
messages = [
|
||||||
{"role": "system", "content": _SYSTEM_PROMPT},
|
{"role": "system", "content": _SYSTEM_PROMPT},
|
||||||
{
|
{
|
||||||
@@ -253,9 +339,20 @@ async def ask_question(body: AskRequest, user: dict = Depends(require_auth)):
|
|||||||
start = (now - timedelta(days=7)).isoformat().replace("+00:00", "Z")
|
start = (now - timedelta(days=7)).isoformat().replace("+00:00", "Z")
|
||||||
end = now.isoformat().replace("+00:00", "Z")
|
end = now.isoformat().replace("+00:00", "Z")
|
||||||
|
|
||||||
query = _build_event_query(entity, start, end)
|
query = _build_event_query(
|
||||||
|
entity,
|
||||||
|
start,
|
||||||
|
end,
|
||||||
|
services=body.services,
|
||||||
|
actor=body.actor,
|
||||||
|
operation=body.operation,
|
||||||
|
result=body.result,
|
||||||
|
include_tags=body.include_tags,
|
||||||
|
exclude_tags=body.exclude_tags,
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
total = events_collection.count_documents(query)
|
||||||
cursor = events_collection.find(query).sort([("timestamp", -1)]).limit(LLM_MAX_EVENTS)
|
cursor = events_collection.find(query).sort([("timestamp", -1)]).limit(LLM_MAX_EVENTS)
|
||||||
events = list(cursor)
|
events = list(cursor)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
@@ -283,7 +380,7 @@ async def ask_question(body: AskRequest, user: dict = Depends(require_auth)):
|
|||||||
llm_error = "LLM_API_KEY is not configured. Set it in your .env to enable AI narrative summarisation."
|
llm_error = "LLM_API_KEY is not configured. Set it in your .env to enable AI narrative summarisation."
|
||||||
else:
|
else:
|
||||||
try:
|
try:
|
||||||
answer = await _call_llm(question, events)
|
answer = await _call_llm(question, events, total=total)
|
||||||
llm_used = True
|
llm_used = True
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
llm_error = f"LLM call failed: {exc}"
|
llm_error = f"LLM call failed: {exc}"
|
||||||
@@ -317,6 +414,7 @@ async def ask_question(body: AskRequest, user: dict = Depends(require_auth)):
|
|||||||
"start": start,
|
"start": start,
|
||||||
"end": end,
|
"end": end,
|
||||||
"event_count": len(events),
|
"event_count": len(events),
|
||||||
|
"total_matched": total,
|
||||||
"mongo_query": json.dumps(query, default=str),
|
"mongo_query": json.dumps(query, default=str),
|
||||||
},
|
},
|
||||||
llm_used=llm_used,
|
llm_used=llm_used,
|
||||||
|
|||||||
@@ -236,7 +236,7 @@ class TestAskEndpoint:
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
async def fake_llm(question, events):
|
async def fake_llm(question, events, total=None):
|
||||||
return "The device had a failed wipe attempt."
|
return "The device had a failed wipe attempt."
|
||||||
|
|
||||||
monkeypatch.setattr("routes.ask.LLM_API_KEY", "fake-key")
|
monkeypatch.setattr("routes.ask.LLM_API_KEY", "fake-key")
|
||||||
@@ -265,7 +265,7 @@ class TestAskEndpoint:
|
|||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
|
||||||
async def failing_llm(question, events):
|
async def failing_llm(question, events, total=None):
|
||||||
raise RuntimeError("LLM service down")
|
raise RuntimeError("LLM service down")
|
||||||
|
|
||||||
monkeypatch.setattr("routes.ask.LLM_API_KEY", "fake-key")
|
monkeypatch.setattr("routes.ask.LLM_API_KEY", "fake-key")
|
||||||
@@ -277,3 +277,76 @@ class TestAskEndpoint:
|
|||||||
assert data["llm_used"] is False # Falls back
|
assert data["llm_used"] is False # Falls back
|
||||||
assert len(data["events"]) == 1
|
assert len(data["events"]) == 1
|
||||||
assert "Found 1 event" in data["answer"]
|
assert "Found 1 event" in data["answer"]
|
||||||
|
|
||||||
|
def test_ask_with_explicit_filters(self, client, mock_events_collection):
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
mock_events_collection.insert_one(
|
||||||
|
{
|
||||||
|
"id": "evt-exchange",
|
||||||
|
"timestamp": now.isoformat(),
|
||||||
|
"service": "Exchange",
|
||||||
|
"operation": "Update",
|
||||||
|
"result": "failure",
|
||||||
|
"actor_display": "Alice",
|
||||||
|
"target_displays": ["LAPTOP-001"],
|
||||||
|
"display_summary": "summary",
|
||||||
|
"raw_text": "raw",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
mock_events_collection.insert_one(
|
||||||
|
{
|
||||||
|
"id": "evt-directory",
|
||||||
|
"timestamp": now.isoformat(),
|
||||||
|
"service": "Directory",
|
||||||
|
"operation": "Add user",
|
||||||
|
"result": "success",
|
||||||
|
"actor_display": "Alice",
|
||||||
|
"target_displays": ["LAPTOP-001"],
|
||||||
|
"display_summary": "summary",
|
||||||
|
"raw_text": "raw",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
response = client.post(
|
||||||
|
"/api/ask",
|
||||||
|
json={"question": "What happened to LAPTOP-001?", "services": ["Exchange"], "result": "failure"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert data["query_info"]["event_count"] == 1
|
||||||
|
assert data["events"][0]["id"] == "evt-exchange"
|
||||||
|
|
||||||
|
def test_ask_with_explicit_actor_filter(self, client, mock_events_collection):
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
mock_events_collection.insert_one(
|
||||||
|
{
|
||||||
|
"id": "evt-bob",
|
||||||
|
"timestamp": now.isoformat(),
|
||||||
|
"service": "Directory",
|
||||||
|
"operation": "Add user",
|
||||||
|
"result": "success",
|
||||||
|
"actor_display": "Bob",
|
||||||
|
"actor_upn": "bob@example.com",
|
||||||
|
"target_displays": ["USER-001"],
|
||||||
|
"display_summary": "summary",
|
||||||
|
"raw_text": "raw",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
mock_events_collection.insert_one(
|
||||||
|
{
|
||||||
|
"id": "evt-alice",
|
||||||
|
"timestamp": now.isoformat(),
|
||||||
|
"service": "Directory",
|
||||||
|
"operation": "Remove user",
|
||||||
|
"result": "success",
|
||||||
|
"actor_display": "Alice",
|
||||||
|
"actor_upn": "alice@example.com",
|
||||||
|
"target_displays": ["USER-001"],
|
||||||
|
"display_summary": "summary",
|
||||||
|
"raw_text": "raw",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
response = client.post("/api/ask", json={"question": "What happened to USER-001?", "actor": "bob"})
|
||||||
|
assert response.status_code == 200
|
||||||
|
data = response.json()
|
||||||
|
assert data["query_info"]["event_count"] == 1
|
||||||
|
assert data["events"][0]["id"] == "evt-bob"
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ services:
|
|||||||
backend:
|
backend:
|
||||||
build: ./backend
|
build: ./backend
|
||||||
# For production, use the pre-built image instead:
|
# For production, use the pre-built image instead:
|
||||||
# image: git.cqre.net/cqrenet/aoc-backend:v1.1.0
|
# image: git.cqre.net/cqrenet/aoc-backend:v1.2.5
|
||||||
container_name: aoc-backend
|
container_name: aoc-backend
|
||||||
restart: always
|
restart: always
|
||||||
env_file:
|
env_file:
|
||||||
|
|||||||
Reference in New Issue
Block a user