feat: initial KosmoConnect platform v0.1

Includes: - Backend services: ingestion (:8001), weather API (:8002), gateway (:8003), billing (:8004) with BTCPay integration - Shared asyncpg pool, TimescaleDB hypertable, Redis, Mosquitto MQTT - React frontend: Dashboard (MapLibre) and Messaging (chat UI) - Bridge daemon for Pi + Meshtastic (Serial/TCP T-Deck support) - Production Docker Compose, Nginx reverse proxy, ops scripts - DEPLOY.md with step-by-step deployment guide
2026-04-12 17:30:15 +02:00
commit 0a4fb7b55e
95 changed files with 9903 additions and 0 deletions
--- a/docs/architecture/data-flow.md
+++ b/docs/architecture/data-flow.md
@@ -0,0 +1,145 @@
+# Data Flow
+
+This document describes how environmental data moves from the sensor to the user's web browser, and how control/commands flow in the opposite direction.
+
+## 1. Sensor Reading & Local Storage
+
+**Frequency**: Every 60 seconds (configurable)
+**Actor**: Enviro-Node firmware
+
+1. MCU wakes from deep sleep (or remains active if interval is short)
+2. Sensors are powered on, stabilized, and read
+3. Raw readings are calibrated and packaged into a compact binary format
+4. The packet is appended to a local ring buffer in SPI flash or SD card
+5. A "data ready" flag is set for the Meshtastic module
+
+### Data Packet Structure (Enviro-Node Local)
+```c
+typedef struct {
+    uint32_t timestamp_unix;
+    int16_t  temperature_c;     // 0.01°C resolution
+    uint16_t humidity_percent;  // 0.01% resolution
+    uint32_t pressure_pa;
+    uint16_t wind_speed_ms;     // 0.1 m/s resolution
+    uint16_t wind_direction;    // degrees
+    uint16_t pm25_ugm3;
+    uint16_t pm10_ugm3;
+    uint16_t gas_resistance_kohm;
+    uint8_t  node_id[6];        // Meshtastic Node ID
+    uint16_t crc16;
+} enviro_packet_t;
+```
+
+## 2. Mesh Transmission
+
+**Frequency**: Every 5-15 minutes (configurable, power-dependent)
+**Actor**: Meshtastic firmware + custom module
+
+1. The custom module requests one or more packets from the local buffer
+2. Packets are encoded into a Meshtastic `DATA` payload on a dedicated environmental channel
+3. The packet is broadcast into the mesh with `want_ack = false` (fire-and-forget for efficiency)
+4. If an infrastructure node is within range (direct or multi-hop), it receives the packet
+5. If no ACK or route is available, the packet remains in the buffer for the next transmission window
+
+### Channel Strategy
+- **Primary Channel**: Standard Meshtastic LongFast for relaying user messages
+- **Secondary Channel**: Custom `KOSMO_ENV` channel for environmental data (can use different frequency slot or SF to avoid congesting primary channel)
+
+## 3. Bridge Ingestion
+
+**Actor**: Infrastructure Node Bridge Daemon
+
+1. The bridge daemon listens to Meshtastic packets via the serial/protobuf API
+2. It filters for packets on the `KOSMO_ENV` channel or with a specific portnum
+3. Valid environmental packets are decoded and wrapped in a JSON envelope:
+   ```json
+   {
+     "type": "enviro_reading",
+     "node_id": "!a1b2c3d4",
+     "received_at": "2026-04-12T09:15:00Z",
+     "hop_count": 2,
+     "payload": {
+       "timestamp": 1744446900,
+       "temperature_c": 18.50,
+       "humidity_percent": 62.30,
+       ...
+     }
+   }
+   ```
+4. The envelope is published to the cloud MQTT broker topic: `kosmo/ingest/enviro`
+
+## 4. Cloud Ingestion
+
+**Actor**: Backend Ingestion Service
+
+1. The ingestion service subscribes to `kosmo/ingest/#`
+2. On receiving a message:
+   - Validate JSON schema
+   - Verify `node_id` is registered and active
+   - Write raw payload to TimescaleDB hypertable `enviro_readings`
+   - Update node `last_seen` timestamp in PostgreSQL
+   - If the node has a backlog, trigger a "sync complete" notification (optional)
+
+### Database Schema (Simplified)
+```sql
+-- TimescaleDB
+CREATE TABLE enviro_readings (
+    time TIMESTAMPTZ NOT NULL,
+    node_id TEXT NOT NULL,
+    temperature_c DOUBLE PRECISION,
+    humidity_percent DOUBLE PRECISION,
+    pressure_pa DOUBLE PRECISION,
+    wind_speed_ms DOUBLE PRECISION,
+    wind_direction SMALLINT,
+    pm25_ugm3 DOUBLE PRECISION,
+    pm10_ugm3 DOUBLE PRECISION,
+    gas_resistance_kohm DOUBLE PRECISION
+);
+SELECT create_hypertable('enviro_readings', by_range('time'));
+
+-- PostgreSQL
+CREATE TABLE nodes (
+    id UUID PRIMARY KEY,
+    mesh_node_id TEXT UNIQUE NOT NULL,
+    name TEXT,
+    location GEOGRAPHY(POINT, 4326),
+    hardware_revision TEXT,
+    installed_at TIMESTAMPTZ,
+    last_seen TIMESTAMPTZ,
+    is_active BOOLEAN DEFAULT true
+);
+```
+
+## 5. Web Dashboard Display
+
+**Actor**: Web Dashboard (React)
+
+1. User loads the dashboard
+2. Frontend queries `/api/v1/weather/latest` and `/api/v1/weather/history`
+3. API service fetches aggregated data from TimescaleDB
+4. Frontend renders:
+   - Map markers with latest readings
+   - Time-series charts (temperature, wind, etc.)
+   - Node health indicators (battery, signal strength, last seen)
+
+## 6. Command Flow (Web to Node)
+
+For configuration updates or remote diagnostics:
+
+1. Admin sends a command via web admin panel (e.g., "change reporting interval to 10 min")
+2. API validates admin permissions
+3. Command is queued in the message gateway for the specific node
+4. Infrastructure node picks up the command via MQTT
+5. Bridge daemon injects the command as a Meshtastic admin packet
+6. Enviro-node receives and applies the config update
+7. Acknowledgment (if requested) flows back through the same path
+
+## Data Retention Policy
+
+| Data Type | Storage Location | Retention |
+|-----------|-----------------|-----------|
+| Raw sensor readings | Enviro-Node flash | 30-90 days (ring buffer) |
+| Ingested readings | TimescaleDB | 2 years raw, then downsampled |
+| Downsampled aggregates | TimescaleDB | Indefinite |
+| Mesh messages | PostgreSQL | 90 days |
+| Audit logs | PostgreSQL | 1 year |
--- a/docs/architecture/messaging-gateway.md
+++ b/docs/architecture/messaging-gateway.md
@@ -0,0 +1,188 @@
+# Messaging Gateway Architecture
+
+The Messaging Gateway is the bridge between the internet (web users) and the Meshtastic mesh. It is the primary monetization surface for the project.
+
+## Business Rules
+
+1. **The mesh is open**: Anyone with a Meshtastic device can join the Kosmo mesh and send/receive messages locally for free.
+2. **The gateway is gated**: Sending a message from the internet to the mesh requires an active subscription.
+3. **Authorization granularity**:
+   - **Network-level**: Subscriber can send to any node reachable through the gateway.
+   - **Node-level**: Subscriber can send only to specific whitelisted nodes (e.g., family members).
+   - Future: **Group-level** access for organizations.
+
+## Gateway Flow: Web → Mesh
+
+```
+User (Web Browser)
+       │
+       ▼
+┌──────────────┐
+│   Web API    │  <-- Validates JWT, checks subscription status
+│   /messages  │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│   Billing    │  <-- Confirms subscriber has active plan & quota remaining
+│    Service   │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│   Message    │  <-- Writes message to outbound queue (RabbitMQ / Redis)
+│   Gateway    │      Topic: `mesh.outbound.{node_id}`
+└──────┬───────┘
+       │ MQTT / TLS
+       ▼
+┌──────────────┐
+│ Infrastructure│ <-- Bridge daemon reads queue
+│    Node       │
+└──────┬───────┘
+       │ Serial / protobuf API
+       ▼
+┌──────────────┐
+│  Meshtastic   │ <-- Broadcasts text message to target node ID
+│    Radio      │
+└───────────────┘
+```
+
+## Gateway Flow: Mesh → Web
+
+Replies and inbound messages from the mesh to a subscriber:
+
+```
+Meshtastic Radio (any node)
+       │
+       ▼
+┌──────────────┐
+│ Infrastructure│ <-- Receives mesh message
+│    Node       │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│   Bridge     │ <-- Publishes to `kosmo/mesh/inbound`
+│   Daemon     │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│   Message    │ <-- Matches sender node ID to subscriber inboxes
+│   Gateway    │
+└──────┬───────┘
+       │
+       ▼
+┌──────────────┐
+│   Web API    │ <-- Stores in user's inbox, sends push notification
+│   Inbox      │
+└───────────────┘
+```
+
+## Subscription Models
+
+### Plan Tiers (Example)
+
+| Tier | Price | Messages/Month | Scope | Features |
+|------|-------|----------------|-------|----------|
+| Free | $0 | 5 (inbound only) | Inbox | Receive replies, view weather |
+| Wanderer | $5/mo | 50 | Network | Send to any node |
+| Guardian | $12/mo | 500 | Node-level | Manage up to 5 linked nodes |
+| Sanctuary | $50/mo | Unlimited | Network + API | Bulk messaging, webhook access |
+
+All paid plans are processed through the Church of Kosmo's self-hosted BTCPay Server at `pay.cqre.net`.
+
+### Authorization Check
+
+When a user attempts to send a message:
+
+```python
+def can_send(user: User, target_node_id: str) -> bool:
+    subscription = user.active_subscription()
+    if not subscription or subscription.is_expired():
+        return False
+    
+    if subscription.plan == "network":
+        return True
+    
+    if subscription.plan == "node_level":
+        return user.allowed_nodes.filter(mesh_node_id=target_node_id).exists()
+    
+    return False
+```
+
+## Message Queue Schema
+
+### Outbound (Cloud → Mesh)
+```json
+{
+  "message_id": "uuid-v4",
+  "sender_user_id": "uuid-v4",
+  "target_node_id": "!a1b2c3d4",
+  "text": "Hello from the web!",
+  "priority": "normal",
+  "max_hops": 7,
+  "want_ack": true,
+  "created_at": "2026-04-12T09:20:00Z",
+  "retry_count": 0
+}
+```
+
+### Inbound (Mesh → Cloud)
+```json
+{
+  "message_id": "uuid-v4",
+  "source_node_id": "!a1b2c3d4",
+  "gateway_node_id": "!gateway01",
+  "text": "Reply from the woods",
+  "hop_count": 3,
+  "rssi": -90,
+  "snr": 8.5,
+  "received_at": "2026-04-12T09:25:00Z"
+}
+```
+
+## Rate Limiting & Anti-Spam
+
+- **Per-user**: Max 1 message per 10 seconds, burst of 5
+- **Per-subscription tier**: Enforced monthly quotas
+- **Per-target-node**: Max 10 web messages per hour (to prevent harassment)
+- **Content filtering**: Basic profanity/spam filter on the gateway
+- **Blocklist**: Users and nodes can block each other
+
+## Delivery Tracking
+
+The gateway tracks message state:
+
+```
+PENDING -> QUEUED -> TRANSMITTED -> DELIVERED (ACK received)
+                          |
+                          +-> FAILED (max retries exceeded)
+```
+
+Users see delivery status in the messaging UI:
+- Single checkmark: Queued
+- Double checkmark: Transmitted by gateway
+- Blue double checkmark: Delivered (ACK from target node)
+
+## Billing Integration
+
+The gateway relies on the **Billing Service** to enforce subscriptions. The billing service:
+- Creates invoices via BTCPay Server Greenfield API
+- Listens to BTCPay webhooks for payment confirmation
+- Manages subscription validity periods and quotas in PostgreSQL
+- Deactivates old subscriptions and resets quotas on successful payment
+
+## Security Considerations
+
+1. **Authentication**: JWT-based auth for web users, API keys for bridge daemons
+2. **Encryption**: Mesh messages are encrypted with the channel key. The bridge daemon does not decrypt content; it only forwards the encrypted payload.
+3. **Privacy**: The gateway logs message metadata (sender, recipient, timestamp, size) but does not log message content.
+4. **Node Impersonation**: Web messages are tagged with a special prefix or sender ID indicating they originated from the gateway, preventing spoofing of local mesh nodes.
+
+## Fallback Behavior
+
+If no infrastructure node is currently online:
+- Outbound messages remain queued for up to 24 hours
+- Users are notified that delivery is delayed
+- If the queue expires, the message is marked as failed and the user's quota is refunded
--- a/docs/architecture/system-overview.md
+++ b/docs/architecture/system-overview.md
@@ -0,0 +1,123 @@
+# System Architecture Overview
+
+## High-Level Concept
+
+KosmoConnect is a **three-tier system**:
+
+1. **Edge Tier**: Solar-powered enviro-nodes running Meshtastic + custom sensor firmware
+2. **Bridge Tier**: Infrastructure nodes with internet backhaul (WiFi/Ethernet/Cellular)
+3. **Cloud Tier**: Central backend services and web frontends
+
+```
+┌─────────────────────────────────────────────────────────────────────────┐
+│                              CLOUD TIER                                 │
+│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐  │
+│  │   Web API    │  │   Ingestion  │  │   Message    │  │   Billing   │  │
+│  │  (Fastify/   │  │   Service    │  │   Gateway    │  │   & Auth    │  │
+│  │   Django)    │  │  (TimescaleDB│  │   (RabbitMQ/ │  │   (Stripe)   │  │
+│  │              │  │   + Redis)   │  │    MQTT)     │  │             │  │
+│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  └──────┬──────┘  │
+│         │                 │                 │                 │         │
+│  ┌──────▼─────────────────▼─────────────────▼─────────────────▼──────┐  │
+│  │                         PostgreSQL                                │  │
+│  │                     (Users, Nodes, Subscriptions)                 │  │
+│  └───────────────────────────────────────────────────────────────────┘  │
+└─────────────────────────────────────────────────────────────────────────┘
+                                    ▲
+                                    │ HTTPS / MQTT over TLS
+                                    ▼
+┌─────────────────────────────────────────────────────────────────────────┐
+│                            BRIDGE TIER                                  │
+│  ┌─────────────────────────────────────────────────────────────────┐    │
+│  │                    Infrastructure Node                           │    │
+│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────────┐  │    │
+│  │  │  Meshtastic │  │   Bridge    │  │   Backhaul (WiFi/Eth/   │  │    │
+│  │  │    Radio    │◄─┤   Daemon    │◄─┤      Cellular)          │  │    │
+│  │  │  (SX1262)   │  │  (Python)   │  │                         │  │    │
+│  │  └─────────────┘  └─────────────┘  └─────────────────────────┘  │    │
+│  └─────────────────────────────────────────────────────────────────┘    │
+│                              (Mains Powered)                            │
+└─────────────────────────────────────────────────────────────────────────┘
+                                    ▲
+                                    │ LoRa / Mesh
+                                    ▼
+┌─────────────────────────────────────────────────────────────────────────┐
+│                             EDGE TIER                                   │
+│  ┌─────────────────────────────────────────────────────────────────┐    │
+│  │                      Enviro-Node                                 │    │
+│  │  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐   │    │
+│  │  │  BME280  │ │  Wind    │ │  Air     │ │   Meshtastic     │   │    │
+│  │  │  (T/H/P) │ │  Sensor  │ │  Quality │ │   Firmware       │   │    │
+│  │  └────┬─────┘ └────┬─────┘ └────┬─────┘ │  + Sensor Module  │   │    │
+│  │       └─────────────┴─────────────┘      │  + Store/Forward  │   │    │
+│  │                   │                      │  + Power Manager  │   │    │
+│  │            ┌──────▼──────┐               └─────────┬─────────┘   │    │
+│  │            │   ESP32/    │                         │             │    │
+│  │            │   nRF52840  │◄────────────────────────┘             │    │
+│  │            └──────┬──────┘                                       │    │
+│  │                   │                                              │    │
+│  │            ┌──────▼──────┐                                       │    │
+│  │            │  Solar +    │                                       │    │
+│  │            │  Battery    │                                       │    │
+│  │            └─────────────┘                                       │    │
+│  └─────────────────────────────────────────────────────────────────┘    │
+│                           (Solar Powered)                               │
+└─────────────────────────────────────────────────────────────────────────┘
+```
+
+## Core Principles
+
+### 1. Open Mesh, Gated Gateway
+The Meshtastic mesh itself is open. Anyone with a compatible device can join, extend range, and benefit from the enviro-node relay infrastructure. However, access to the **web-to-mesh gateway** (sending messages from the internet to the mesh) is restricted to paying subscribers.
+
+### 2. Store-and-Forward Data Offload
+Enviro-nodes collect data continuously but may not always have a direct route to an infrastructure node. Data is buffered in local flash/SD and transmitted when a route becomes available. The Meshtastic store-and-forward module may be leveraged or extended.
+
+### 3. Separation of Concerns
+- **Meshtastic handles**: Mesh routing, encryption, device-to-device messaging, channel management
+- **Custom firmware handles**: Sensor reading, power management, data buffering, packet formatting
+- **Backend handles**: User auth, subscription billing, data persistence, web APIs, message queuing
+- **Bridge handles**: Protocol translation between Meshtastic protobufs and cloud MQTT/HTTPS
+
+## Component Boundaries
+
+### Enviro-Node (Edge)
+**Hardware**: Custom PCB based on ESP32-S3-WROOM-1 or nRF52840 + SX1262, sensor headers, solar charge controller, battery management.
+**Firmware**: Either a Meshtastic firmware fork with a custom sensor module, or a companion MCU architecture where Meshtastic runs on one chip and a sensor controller runs on another.
+
+### Infrastructure Node (Bridge)
+**Hardware**: Meshtastic device (LILYGO T-Beam, RAK4631, or custom) with reliable internet backhaul.
+**Software**: A bridge daemon running alongside the Meshtastic firmware (via serial/API) that forwards environmental data to the cloud and injects outbound mesh messages from the cloud queue.
+
+### Central Backend (Cloud)
+- **Ingestion Service**: Consumes MQTT from infrastructure nodes, validates, writes to TimescaleDB
+- **API Service**: REST/GraphQL API for weather data, node registry, health status
+- **Message Gateway**: Manages the queue of web-to-mesh messages, handles delivery confirmations, rate limiting
+- **Billing & Auth**: Stripe integration for subscriptions, OAuth2/JWT for user auth, node-level permission checks
+
+### Web Frontend (Cloud)
+- **Dashboard**: Map-based weather visualization, node health, historical charts
+- **Messaging Client**: Compose messages to mesh nodes by node ID or alias, view replies
+- **Admin Panel**: Node onboarding, subscriber management, network diagnostics
+
+## Technology Stack Recommendations
+
+| Layer | Technology |
+|-------|------------|
+| Enviro-Node MCU | ESP32-S3 (for power/performance) or nRF52840 (for efficiency) |
+| Radio | Semtech SX1262 (Meshtastic standard) |
+| Sensors | BME680 (T/H/P/Gas), SPS30 (PM), Davis anemometer (wind) |
+| Bridge Daemon | Python with `meshtastic` CLI library + `paho-mqtt` |
+| Backend Runtime | Python (FastAPI) or Node.js (NestJS) |
+| Database (Time-series) | TimescaleDB or InfluxDB |
+| Database (Relational) | PostgreSQL |
+| Message Queue | RabbitMQ or Redis Streams |
+| Frontend | React / Vue + MapLibre GL |
+| Infra | Docker, Terraform, Ansible |
+
+## Scalability Considerations
+
+- A single infrastructure node can serve a large mesh area, but dense networks benefit from multiple infrastructure nodes for redundancy.
+- Environmental data is small and infrequent (e.g., one packet every 5-15 minutes), so bandwidth is not a concern.
+- Web-to-mesh messaging is low bandwidth but requires delivery tracking and rate limiting to prevent spam.
+- The system should gracefully degrade if the cloud is unreachable; the mesh continues to function locally.