Skip to content

SCRUM-233 — Dashboard PromQL template catalogue and Prometheus client

Plan ref: DSH-C3/C4 (docs/11-admin-plane-plan.md). Stacked on SCRUM-235.

What exists

Piece Notes
internal/promql the only place PromQL is written. Lookup(name) → ErrUnknownTemplate; Template.Build(service, window, allowed) checks the service against the Prometheus allow-list and ^[a-z][a-z0-9_-]{0,31}$ before substitution → ErrUnknownService (both map to 400 in the handler tickets). Window rendered as whole seconds, min 60 s
Templates rps_by_status, error_ratio, latency_p50/p95/p99, cpu, memory (cAdvisor, by compose service), goroutines, heap_inuse, gc_pause_max (the default Go collector publishes a GC summary, so p99 is not available; quantile="1" is the max)
internal/prometheus.Client implements source.MetricsSource: Instant, Range, Targets over /api/v1/query, /query_range, /targets; per-call timeout = DASHBOARD_UPSTREAM_TIMEOUT; 16 MiB body cap; errors carry Prometheus's errorType: error, never the URL
Columnar decode full from..to grid preallocated, each series pre-filled with NaN, points written at (t-from)/step; off-grid points ignored
AllowList services from active targets' service label, cached 30 s, one refresh in flight; a failed refresh keeps the last good set and is retried after 2 s

Series selection relies on a service label on every scrape target (SCRUM-230) and on services naming metrics <svc>_http_requests_total{route,method,status} / <svc>_http_request_duration_seconds_bucket.

How to verify

cd services/dashboard
gofmt -l . && go vet ./... && go test -race -count=1 ./...

Tests: every template builds; no placeholder survives; injection attempts in the service name (quote, brace, newline, not allow-listed) are refused; golden strings; recorded fixtures (internal/prometheus/testdata/) for matrix with a gap (NaN alignment), vector, scalar, error body, targets; body cap; timeout; allow-list caching, single refresh under concurrency, last-good-set on error, quick retry after an error, refresh surviving the starting request's cancellation.

Results at time of writing

  • gofmt, go vet, go test -race (7 packages): pass.

How it was built

DeepSeek run scoped (Landlock) to services/dashboard (187 s, ~37k output tokens, reasoning effort low). Claude review fixed the allow-list refresh: it ran on the first caller's context and cached failures for the full 30 s, so one cancelled request or a Prometheus blip made every service "unknown" for half a minute. Two regression tests added.