Scrape Ledgerline's metrics from falken through czy's Traefik

czy publishes no service ports, so Traefik exposes each service's :9464
as https://ledgerline.vione.cloud/metrics/<service> behind basic auth,
and the platform's Prometheus gets a `ledgerline` job with one target
per service. The targets share an address; a `service` label tells them
apart, and relabels derive __metrics_path__ and instance from it.

create-secrets.sh now also creates the prometheus-scrape-auth secret
(mounted optional, so a missing secret fails only this job) and writes
the gitignored htpasswd file Traefik reads, which is copied to czy like
.env.observability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Movses Aslanyan
2026-10-19 20:31:10 +03:30
co-authored by Claude Opus 5.5
parent 920712f32b
commit b3e7c14958
7 changed files with 147 additions and 17 deletions
+20 -3
View File
@@ -9,9 +9,14 @@
# obs-basic-auth htpasswd for the Traefik basic-auth middleware
# (Prometheus, Alertmanager, Loki, Tempo, Alloy UI, OTLP)
# grafana-admin Grafana admin user/password
# and writes .env.observability at the repo root (gitignored) for the czy-side
# Alloy log agent and the OTLP auth header of instrumented services; copy it to
# the server after every regeneration (see k8s/README.md).
# prometheus-scrape-auth
# username/password Prometheus sends when scraping Ledgerline
# and writes two gitignored files for czy; copy both to the server after every
# regeneration (see k8s/README.md):
# .env.observability for the Alloy log agent and the OTLP auth header of
# instrumented services
# observability/traefik/metrics-users
# htpasswd for Traefik's basic auth on /metrics/<service>
#
# KUBECTL defaults to `kubectl`; e.g. KUBECTL="ssh falken kubectl" to run it on
# the node when the API server isn't reachable from here.
@@ -48,6 +53,11 @@ kubectl create secret generic grafana-admin -n "$NS" \
--from-literal=admin-password="$GRAFANA_ADMIN_PASSWORD" \
--dry-run=client -o yaml | $KUBECTL apply -f -
kubectl create secret generic prometheus-scrape-auth -n "$NS" \
--from-literal=username="$OBS_USER" \
--from-literal=password="$OBS_PASSWORD" \
--dry-run=client -o yaml | $KUBECTL apply -f -
# OTLP header for instrumented services: URL-encoded (space -> %20), and
# newlines stripped because GNU base64 wraps at 76 chars and this is ~80.
OTLP_BASIC=$(printf '%s:%s' "$OBS_USER" "$OBS_PASSWORD" | base64 | tr -d '\n')
@@ -64,3 +74,10 @@ OTEL_EXPORTER_OTLP_HEADERS=Authorization=Basic%20$OTLP_BASIC
EOF
)
echo "wrote $(cd .. && pwd)/.env.observability"
(
umask 077
mkdir -p ../observability/traefik
htpasswd -nbB "$OBS_USER" "$OBS_PASSWORD" > ../observability/traefik/metrics-users
)
echo "wrote $(cd .. && pwd)/observability/traefik/metrics-users"