feat: implement Phase 3 scaling
Some checks failed
CI / lint-and-test (push) Has been cancelled

- Replace skip-based pagination with cursor-based pagination (timestamp|_id cursors)
- Add Prometheus /metrics endpoint with request latency, fetch volume, and error counters
- Implement incremental fetch watermarking per source (watermarks collection in MongoDB)
- Add Graph change notification webhook endpoint (/api/webhooks/graph)
- Add correlation ID middleware for distributed tracing (x-request-id header)
- Update frontend to use cursor-based pagination with Prev/Next navigation
- Update tests for cursor pagination, metrics, webhooks, and watermark mocking
This commit is contained in:
2026-04-14 14:58:50 +02:00
parent 9271b4e461
commit b0198012eb
17 changed files with 402 additions and 147 deletions

16
backend/middleware.py Normal file
View File

@@ -0,0 +1,16 @@
import uuid
import structlog
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
class CorrelationIdMiddleware(BaseHTTPMiddleware):
"""Inject or propagate a correlation ID for every request."""
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response:
cid = request.headers.get("x-request-id") or uuid.uuid4().hex
structlog.contextvars.bind_contextvars(correlation_id=cid)
response = await call_next(request)
response.headers["x-request-id"] = cid
return response