[SIMPL-30451] Assess the tightly coupled path against the sandbox

T2, T3, T5 and T8 are confirmed from the live dagster-instance ConfigMap and from dagster-role. None of the T checks could be executed: the tightly coupled path needs this code location registered on the platform Dagster, and unlike the pipes transport it cannot run from a standalone pod, because every step pod connects to the metadata database itself and so they must share instance storage rather than a local SQLite file.

C6 fails on that cluster and explains why no run has ever executed there. The launcher sends run pods to namespace dagster while its instance_config_map, postgres_password_secret and dagster-shared-pvc all live in dataprovider01; those references do not cross namespaces. dataprovider01 holds no pods labelled dagster/run-id and the daemon log shows no launch activity. C6 now tells the reader to check the launcher's namespace-local dependencies, not just the namespace.

Changelog: changed

documents/user-guide/readiness-checklist.md

@@ -9,9 +9,10 @@ location image.
 
 > **L4–L10 were cleared on sandbox-cat-dat on 2026-08-31.** Checks L1–L3 and L11,
 > which exercise the payload contract, the message-parsing path and image-level
-> isolation, are verified locally. What remains unverified on a cluster is the
-> **tightly coupled** Kubernetes path — the T rows and the Kubernetes rows of
-> section 4.3 — which uses `k8s_job_executor` rather than the pipes client.
+> isolation, are verified locally. The **tightly coupled** Kubernetes path has
+> been assessed against the same cluster but not executed — T2, T3, T5 and T8
+> are confirmed from the live instance configuration and C6 fails there. See
+> [section 2.1](#21-sandbox-state-2026-08-31).
 
 ---
 
@@ -23,8 +24,8 @@ location image.
 | C2 | Jobs are registered | Dagster UI → **Jobs** | The jobs listed in the guide's section 5.5 appear under the code location |
 | C3 | Image tag matches the intended release | `kubectl -n dagster get deploy -l dagster/code-location=distributed-execution -o jsonpath='{.items[*].spec.template.spec.containers[*].image}'` | Tag equals the version in `pipeline.variables.sh`; never `latest` |
 | C4 | Image architecture matches the nodes | `docker manifest inspect <image>` | Includes `linux/amd64`; a manifest with only `linux/arm64` produces `no match for platform` at pull time |
-| C5 | Run launcher type is as intended | `kubectl -n dagster get cm dagster-instance -o yaml` | `run_launcher` block shows `K8sRunLauncher` and the expected `job_namespace` |
-| C6 | Target namespace exists and is schedulable | `kubectl get ns <namespace>` and `kubectl -n <namespace> get resourcequota` | Namespace is `Active`; remaining quota exceeds the job's aggregate requests |
+| C5 | Run launcher type is as intended | `kubectl -n <namespace> get cm dagster-instance -o yaml` | `run_launcher` block shows `K8sRunLauncher` and the expected `job_namespace` |
+| C6 | Target namespace exists and is schedulable | `kubectl get ns <namespace>` and `kubectl -n <namespace> get resourcequota` | Namespace is `Active`; remaining quota exceeds the job's aggregate requests. Also confirm the launcher's `instance_config_map`, `postgres_password_secret` and any PVC volumes exist **in that same namespace** — these references do not cross namespaces, and on sandbox-cat-dat they do not match `job_namespace` (see section 2.1) |
 
 ## 2. Tightly coupled checks
 
@@ -39,6 +40,43 @@ location image.
 | T7 | Step pod egress is permitted | Same run | Steps do not hang in `STARTING`; run logs contain no connection timeouts to port 5432 |
 | T8 | Failure surfaces as a pod failure | Force a step failure in a scratch namespace | `failPodOnRunFailure: true` is set, and the step pod reports `Failed` rather than `Completed` |
 
+### 2.1 Sandbox state, 2026-08-31
+
+The T checks were assessed against sandbox-cat-dat by reading the live
+`dagster-instance` ConfigMap in `dataprovider01`. None of them could be *executed*:
+the tightly coupled Kubernetes path needs this code location registered on the
+platform Dagster, which is blocked on the control plane skew, and unlike the
+pipes transport it cannot be exercised from a standalone pod — every step pod
+connects to the metadata database on its own, so they must share instance
+storage rather than a local SQLite file.
+
+What the configuration shows:
+
+| # | Finding |
+|---|---|
+| C5 | `run_launcher` is `K8sRunLauncher` with `job_namespace: dagster` |
+| T2 | Vault injection is configured — `pod_template_spec_metadata` carries the banzaicloud annotations with role `sandbox-cat-dat-role` |
+| T3 | `S3_ENDPOINT_URL` is `https://s3.sandbox-cat-dat.simpl-europe.eu`, with access keys injected from Vault |
+| T5 | Passes. `dagster-role`, bound to `dagster-svc-account`, grants `batch/jobs` with create, delete, get, list, patch, update and watch |
+| T8 | `fail_pod_on_run_failure: true` is set |
+
+**C6 fails, and it is the reason nothing has ever run here.** The launcher sends
+run pods to namespace `dagster`, but every namespace-local dependency it names —
+`instance_config_map: dagster-instance`, `postgres_password_secret:
+dagster-postgresql-secret`, and the `dagster-shared-pvc` volume — exists in
+`dataprovider01`. A ConfigMap, Secret or PVC reference resolves only within the
+pod's own namespace, so a run pod placed in `dagster` would reference three
+objects that are not there. Consistent with that, `dataprovider01` holds no pods
+labelled `dagster/run-id` and the daemon log shows no launch activity.
+
+One caveat on the evidence: project-scoped access means `kubectl get ns dagster`
+returns `Forbidden` rather than `NotFound`, so whether that namespace exists is
+unconfirmed. It does not change the conclusion — the dependencies are in the
+wrong namespace either way.
+
+This is a platform configuration defect, not a defect in this service. It is
+exactly what C6 exists to catch.
+
 ## 3. Loosely coupled checks
 
 | # | Check | How to verify | Expected evidence |