Capacity Planning

Architecture Summary

Reva consists of six components that share host resources:

                     +-----------+
Microsoft Teams ---->| Reva App  |----> Ollama (GPU)
                     |  FastAPI  |       - Router: llama3.2:3b
                     |  Python   |       - Agent:  qwen3:14b
                     +-----+-----+
                           |
                +----------+----------+
                |          |          |
          +-----+--+ +----+---+ +----+---+
          |PostgreSQL| | Redis  | |  MCP   |
          | pgvector | |  cache | | servers|
          +----------+ +--------+ +--------+
                                    (Release, Jira)

Resource consumption profile:

• GPU (Ollama): The dominant bottleneck. All LLM inference is serialized through the GPU. Every user request requires 2–4 LLM calls (1 router classification + 1–3 agent steps).
• CPU/RAM (Reva App): Lightweight async Python process. Spends most time waiting on GPU and MCP responses.
• CPU/RAM (MCP servers): Two sidecar containers handling API calls to Digital.ai Release and Jira. Median latency 63ms.
• CPU/RAM (PostgreSQL): Conversation history and pgvector embeddings. Sub-millisecond query times. Never a bottleneck in testing.
• CPU/RAM (Redis): Session cache. Sub-millisecond latency. Negligible resource usage.

Component Requirements

Minimum resources per component for a single-instance deployment:

GPU Sizing

The GPU is the single most important capacity decision. All other components are cheap relative to the GPU.

Key Findings from Testing (RTX 5070 Ti 16GB)

• Single user: 22s median end-to-end (4 agent steps, 1 tool call).
• Throughput ceiling: 0.05 QPS (constant regardless of concurrency) = ~3 requests/min.
• Linear degradation: Each additional concurrent user adds ~20s to wait time due to GPU queue serialization.
• 5 concurrent users: 40% of requests timed out at the 120s client timeout.
• NUM_PARALLEL=2 on 16GB: 5x slower (110s vs 22s). The qwen3:14b model uses ~9 GB for weights, leaving only ~7 GB for KV cache. Two parallel sequences overflow VRAM and trigger CPU offloading.

When to Use Which GPU Tier

• 8 GB: Proof of concept or personal use. Must use a smaller model (qwen3:8b). Expect reduced tool-calling accuracy.
• 16 GB: Small team (< 20 users), light usage. Acceptable if most users send only 1–2 queries per day.
• 24 GB: Medium team (20–100 users), moderate usage. Enables NUM_PARALLEL=2 without KV cache overflow. First tier where vLLM becomes viable.
• 48 GB: Large deployment or heavy usage. vLLM continuous batching provides near-linear throughput scaling up to 10+ concurrent users.

Users per Instance

“Concurrent users” is different from “total users.” Most enterprise users interact with Reva a few times per day. The table below maps total Teams users to required infrastructure, based on measured throughput of ~3 requests/min on a 16GB GPU.

How to Estimate Your Usage Pattern

1. Count your total Reva-eligible users (release managers, ops engineers, etc.).
2. Estimate peak hour load: typically 10–20% of users are active in the busiest hour.
3. Multiply active users by average queries per hour.
4. If peak queries/minute exceeds 3 (16GB) or 6 (24GB), you need a larger GPU or multiple instances.

Scaling Decision Tree

Use this tree when response times or timeouts indicate scaling is needed:

Scaling Options, in Priority Order

K8s Resource Examples

Production-tested resource specifications from the project’s Kubernetes manifests.

Reva Application Pod (includes MCP sidecars)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: reva
  namespace: reva
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: reva
          image: reva:latest
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              memory: 1Gi

        - name: release-mcp
          image: xebialabsearlyaccess/dai-release-mcp:25.3.0-beta.926
          resources:
            requests:
              cpu: 50m
              memory: 128Mi
            limits:
              memory: 256Mi

        - name: jira-mcp
          image: ghcr.io/sooperset/mcp-atlassian:0.21.0
          resources:
            requests:
              cpu: 50m
              memory: 128Mi
            limits:
              memory: 256Mi

PostgreSQL StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: reva
spec:
  template:
    spec:
      containers:
        - name: postgres
          image: pgvector/pgvector:pg16
          resources:
            requests:
              cpu: 100m
              memory: 256Mi
            limits:
              memory: 512Mi
  volumeClaimTemplates:
    - metadata:
        name: pgdata
      spec:
        resources:
          requests:
            storage: 10Gi

Redis

apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: reva
spec:
  template:
    spec:
      containers:
        - name: redis
          image: redis:7-alpine
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
            limits:
              memory: 128Mi

Total Pod Resource Budget

Cloud LLM Comparison

When local GPU capacity is insufficient, cloud LLM APIs offer unlimited scaling at per-token cost. This comparison assumes the Ollama router model (llama3.2:3b) still runs locally.

Per-Request Cost Estimate

A typical Reva request involves ~4 LLM calls with approximately 4,000 input tokens and 800 output tokens total.

Break-Even Analysis

GPU amortization cost over 3 years (typical enterprise hardware lifecycle):

When Cloud Makes Sense

• Fewer than 100 requests/day and you want to avoid GPU procurement
• Burst capacity needed beyond what local GPU can handle
• Proof of concept or trial deployments

When Local GPU Makes Sense

• More than 100 requests/day sustained
• Data sovereignty requirements (no data leaves the network)
• Predictable, flat monthly cost preferred over variable per-token billing

Monitoring for Capacity

Reva exposes metrics via GET /api/stats (JSON) and GET /api/metrics (Prometheus). These are the capacity-relevant metrics:

Prometheus Alerting Rules (Example)

groups:
  - name: reva-capacity
    rules:
      - alert: RevaHighLatency
        expr: reva_request_duration_seconds{quantile="0.95"} > 30
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "Reva p95 response time exceeds 30s"

      - alert: RevaVeryHighLatency
        expr: reva_request_duration_seconds{quantile="0.95"} > 60
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Reva p95 response time exceeds 60s"

      - alert: RevaHighErrorRate
        expr: rate(reva_requests_total{status="error"}[5m]) / rate(reva_requests_total[5m]) > 0.05
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "Reva error rate exceeds 5%"

      - alert: RevaDBPoolExhaustion
        expr: reva_db_pool_checked_out / reva_db_pool_size > 0.9
        for: 1m
        labels:
          severity: warning
        annotations:
          summary: "Database connection pool > 90% utilized"

Manual Capacity Check

Run this periodically or after scaling changes:

# Quick capacity snapshot
curl -s http://localhost:3978/api/stats | jq '{
  response_p50: .request_performance.response_time_p50_s,
  response_p95: .request_performance.response_time_p95_s,
  rpm: .request_performance.requests_per_minute,
  active_sessions: .conversations.active_sessions,
  llm_p50: .llm.response_time_p50_s,
  db_pool_used: .infrastructure.db_pool_checked_out,
  db_pool_max: (.infrastructure.db_pool_size + 20),
  process_rss_mb: (.infrastructure.process_rss_bytes / 1048576 | floor)
}'

Storage Planning

PostgreSQL Growth

PostgreSQL stores conversation history, pgvector embeddings, and metadata. Growth depends on usage volume.

Estimated Monthly Growth

The default PVC size of 10 Gi is sufficient for all but the heaviest deployments over multiple years. Factor in daily compressed backups (~5–10% of DB size per backup, 30-day retention):

Ollama Model Storage

Model files are stored on the Ollama host (not in the K8s cluster).

Docker Log Storage

Log rotation is configured in docker-compose.yml: