SCRUM-231 — CI validation of the observability configs¶
Plan ref: DSH-A5 (docs/11-admin-plane-plan.md). Stacked on SCRUM-230.
What exists¶
| Piece | Notes |
|---|---|
ci/scripts/observability-checks.sh |
reads the Prometheus, Loki and Alloy image references from deploy/compose.yaml (validator = deployed version) and runs, in those images: promtool check config, alloy fmt (fails if the output differs from the file, so unformatted files fail too), loki -verify-config; stops at the first failure with the tool's own error |
ci/services/Jenkinsfile |
stage "Observability configs", dashboard job only, before Test, on the agent (the go container has no docker CLI) |
ci/dispatcher/Jenkinsfile |
a change under deploy/observability/ dispatches otomo-dashboard (and only it) — there is no Jenkins job for deploy/ |
ci/scripts/test-observability-checks.sh |
stub docker: asserts the images are compose's, a failing promtool fails, an alloy fmt diff fails |
deploy/observability/config.alloy |
reformatted with alloy fmt (tabs): the new check caught that the committed file was not in canonical form |
How to verify¶
sh ci/scripts/test-observability-checks.sh
sh ci/scripts/observability-checks.sh # on a host with docker
Results at time of writing¶
- Stub test: pass.
- The real script driven through a shim that maps each
docker runto the local promtool 3.15 / alloy v1.20 / loki 3.7.8 binaries: all three OK; with a brokenscrape_intervalappended to prometheus.yml it exits 1. - Groovy was re-read, not executed (no Jenkins locally). The Jenkins webhook is currently unreachable (see HANDOVER), so this has not run on the real server yet.
How it was built¶
DeepSeek run scoped (Landlock) to ci (110 s, ~19k output tokens, reasoning effort
low); it reported the unformatted alloy file. Claude reformatted it and ran the
script against the real validators, including the failing case.