feat: Redis caching + async queue for LLM scaling (v1.6.0)
- Add async Redis client singleton (redis_client.py) for caching and arq pool
- Add arq job functions (jobs.py) for background LLM processing
- Cache ask/explain LLM responses with TTL (1h ask, 24h explain)
- Add async mode to /api/ask: enqueue job, return job_id, poll /api/jobs/{id}
- Add GET /api/jobs/{job_id} endpoint for job status polling
- Add arq worker service to docker-compose (dev + prod)
- Switch from Redis to Valkey (BSD fork) in Docker Compose
- Add REDIS_URL config setting
- Add tests for cache hit, async mode, and job status
This commit is contained in:
+4
-3
@@ -65,9 +65,10 @@ Goal: add AI-powered analysis and external tool integration.
|
||||
- [x] AI feature flag (`AI_FEATURES_ENABLED`) to gate LLM-dependent features
|
||||
- [x] Natural language query endpoint (`/api/ask`) with intent extraction and smart sampling
|
||||
- [x] MCP (Model Context Protocol) server for Claude Desktop / Cursor integration
|
||||
- [x] Valkey caching for LLM responses and frequent queries
|
||||
- [x] Async queue (arq) for LLM requests to prevent timeout/cost explosions at scale
|
||||
- [ ] Advanced analytics dashboard (trending operations, anomaly detection)
|
||||
- [ ] Redis caching for LLM responses and frequent queries
|
||||
- [ ] Async queue for LLM requests to prevent timeout/cost explosions at scale
|
||||
|
||||
## Completed in this PR
|
||||
All Phase 5 items marked done were implemented in v1.3.0.
|
||||
All Phase 5 items marked done were implemented in v1.3.0–v1.5.0.
|
||||
Redis caching + async queue implemented in v1.6.0, switched to Valkey.
|
||||
|
||||
Reference in New Issue
Block a user