Scrape Ledgerline's metrics from falken through czy's Traefik

czy publishes no service ports, so Traefik exposes each service's :9464
as https://ledgerline.vione.cloud/metrics/<service> behind basic auth,
and the platform's Prometheus gets a `ledgerline` job with one target
per service. The targets share an address; a `service` label tells them
apart, and relabels derive __metrics_path__ and instance from it.

create-secrets.sh now also creates the prometheus-scrape-auth secret
(mounted optional, so a missing secret fails only this job) and writes
the gitignored htpasswd file Traefik reads, which is copied to czy like
.env.observability.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

k8s/prometheus/deployment.yaml

@@ -48,6 +48,9 @@ spec:
               mountPath: /etc/prometheus/rules
             - name: data
               mountPath: /prometheus
+            - name: scrape-auth
+              mountPath: /etc/prometheus/secrets
+              readOnly: true
       volumes:
         - name: config
           configMap:
@@ -58,3 +61,10 @@ spec:
         - name: data
           persistentVolumeClaim:
             claimName: prometheus-data
+        # Basic-auth credentials for the `ledgerline` scrape job, created by
+        # create-secrets.sh. Optional so a missing secret costs that job's
+        # scrapes (up == 0) rather than the whole Prometheus pod.
+        - name: scrape-auth
+          secret:
+            secretName: prometheus-scrape-auth
+            optional: true

k8s/prometheus/prometheus.yml

@@ -35,7 +35,38 @@ scrape_configs:
     static_configs:
       - targets: ["grafana:3000"]
 
-  # Application scrape jobs go here -- the equivalent of the Ansible role's
-  # `prometheus_extra_scrape_jobs`. Ledgerline runs on czy with its service
-  # ports unpublished, so a reachable path to /metrics has to be decided first
-  # (Phase 5). Re-apply with `kubectl apply -k k8s/` after editing.
+  # Application scrape jobs -- the equivalent of the Ansible role's
+  # `prometheus_extra_scrape_jobs`. Re-apply with `kubectl apply -k k8s/`.
+  #
+  # Ledgerline runs on czy with no service ports published. Each service
+  # serves metrics on its own :9464, which czy's Traefik exposes as
+  # https://ledgerline.vione.cloud/metrics/<service> behind basic auth
+  # (docker-compose.traefik.yml). All seven targets share one address, so the
+  # `service` label is what tells them apart; the relabels derive the path
+  # and a readable instance from it.
+  - job_name: ledgerline
+    scheme: https
+    basic_auth:
+      username_file: /etc/prometheus/secrets/username
+      password_file: /etc/prometheus/secrets/password
+    static_configs:
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: gateway }
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: ledger }
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: fraud }
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: sanctions }
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: insights }
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: notifier }
+      - targets: ["ledgerline.vione.cloud"]
+        labels: { service: audit }
+    relabel_configs:
+      - source_labels: [service]
+        target_label: __metrics_path__
+        replacement: /metrics/$1
+      - source_labels: [service]
+        target_label: instance