Merge pull request 'Deploy to sandbox: develop --> main' (#2) from develop into main

Reviewed-on: #2

CHANGELOG.md

@@ -4,15 +4,33 @@
 
 - Repository skeleton for the `distributed-execution` supporting data service.
 - User guide covering execution-target selection, decision support and readiness checks.
+- Cluster-configuration input reference: mandatory versus optional inputs per setup,
+  a reusable minimum input set and worked examples in both setups' own formats.
+- Requirement traceability mapping the guide's sections onto SIMPL-30451 and SIMPL-30787.
 - Runnable tightly coupled reference implementations (`in_process`, `multiprocess`, `k8s_job_executor`).
 - Runnable loosely coupled reference implementations via dagster-pipes
   (`PipesSubprocessClient`, `PipesK8sClient`) with a standalone external payload image.
 - Working example configuration and RBAC for both execution targets.
 
+### verified
+
+- `tightly_coupled_k8s_job` executed end to end on sandbox-cat-dat through the
+  platform `K8sRunLauncher` (run `1d8cb167-9fb8-4a34-a036-789eac381e13`), closing
+  readiness rows T6 and T7. Step outputs are written to the shared
+  `dagster-shared-pvc` by pinning `fs_io_manager`'s `base_dir`; `DAGSTER_HOME` set
+  through `step_k8s_config` is ignored, because the Dagster chart injects
+  `DAGSTER_HOME=/tmp/dagster` ahead of it.
+- `loosely_coupled_k8s_job` reached `RUN_SUCCESS` on sandbox-cat-dat through the
+  standalone probe (run `cff9b348-bfc3-4ac1-ab51-a94892b8e3a0`), with four
+  external payload pods reporting over the pod-log message channel.
+- Tightly coupled preflight evidence now checks the chart's actual
+  `DAGSTER_PG_PASSWORD` injection without exposing its value.
+- Dagster UI screenshots cover the registered jobs, graph, code-configured
+  execution-target tags and a successful reference run.
+- The platform deployment diagram includes the service and both integration
+  patterns.
+
 ### pending
 
-- Publish the payload image to the container registry.
-- End-to-end run of `loosely_coupled_k8s_job` against a cluster, and confirmation
-  of the Kubernetes rows in the readiness checklist.
-- Screenshots of UI surfaces and integration-pattern diagrams.
-- Platform architecture document update.
+- Extend the project-level GitLab pipeline to publish the payload image as well
+  as the code-location image.

README.md

@@ -14,7 +14,7 @@ actually run, and how does runtime information get back to the control plane?*
 | User guide | [documents/user-guide/distributed-execution-guide.md](documents/user-guide/distributed-execution-guide.md) | Complete |
 | Readiness checklist | [documents/user-guide/readiness-checklist.md](documents/user-guide/readiness-checklist.md) | Complete |
 | Tightly coupled reference | [src/distributed_execution/tightly_coupled/jobs.py](src/distributed_execution/tightly_coupled/jobs.py) | Runnable |
-| Loosely coupled reference | [src/distributed_execution/loosely_coupled/jobs.py](src/distributed_execution/loosely_coupled/jobs.py) | Runnable (subprocess verified) |
+| Loosely coupled reference | [src/distributed_execution/loosely_coupled/jobs.py](src/distributed_execution/loosely_coupled/jobs.py) | Runnable and cluster-verified |
 | External payload | [payload/work.py](payload/work.py) | Runnable |
 | Example configuration | [yaml/](yaml/) | Complete |
 
@@ -35,7 +35,7 @@ distributed-execution/
 │   ├── work.py
 │   ├── requirements.txt
 │   └── Dockerfile
-├── documents/user-guide/           # AC1-AC4 documentation
+├── documents/user-guide/           # Guide, checklist and requirement traceability
 ├── yaml/                           # Working example configuration
 ├── tests/
 ├── Dockerfile
@@ -88,10 +88,10 @@ pipeline still has to learn.
 
 ## Status
 
-Both execution targets are implemented. The tightly coupled jobs and the loosely
-coupled **subprocess** transport are verified by the test suite. The loosely
-coupled **Kubernetes** transport is implemented but has not yet been run against
-a cluster; see the guide's *Outstanding work* section.
+Both execution targets are implemented. Local variants are verified by the test
+suite, `tightly_coupled_k8s_job` has run end to end through the platform launcher,
+and `loosely_coupled_k8s_job` has run end to end through the standalone cluster
+probe. See the guide and readiness checklist for the recorded evidence.
 
 ## Licence
 

documents/user-guide/distributed-execution-guide.md

@@ -5,14 +5,10 @@
 integration pattern for a Dagster workflow on the Simpl orchestration platform.
 
 > **Document status.** Both setups are backed by runnable reference
-> implementations in this repository. The tightly coupled setup and the loosely
-> coupled **subprocess** transport are verified end to end, including runs
-> launched from the Dagster UI — see [section 5.6](#56-what-a-verified-run-actually-produced).
-> The loosely coupled **Kubernetes** transport was run on sandbox-cat-dat on
-> 2026-08-31 and reached `RUN_SUCCESS`; see the readiness checklist for the
-> evidence. What remains unproven on a cluster is the *tightly* coupled
-> Kubernetes path — `k8s_job_executor` — whose rows stay marked
-> **NOT CLUSTER-VERIFIED**. See [Outstanding work](#8-outstanding-work).
+> implementations and verified end to end. The loosely coupled Kubernetes path
+> reached `RUN_SUCCESS` on sandbox-cat-dat on 2026-08-31; the tightly coupled
+> `k8s_job_executor` path reached `RUN_SUCCESS` through the platform launcher on
+> 2026-09-01. See the readiness checklist for the recorded evidence.
 
 ---
 
@@ -70,7 +66,7 @@ pattern.
 
 ---
 
-## 2. AC1 — Choosing an execution target
+## 2. Choosing an execution target and cluster setup
 
 ### 2.1 Choose tightly coupled when
 
@@ -125,7 +121,7 @@ before it is run for the first time. Section 5.4 explains where each item lives.
 
 ---
 
-## 3. AC2 — Decision support
+## 3. Decision support and cluster-configuration inputs
 
 ### 3.1 Side-by-side comparison
 
@@ -183,9 +179,134 @@ Use these as a fast triage. If several point the same way, that is your answer.
   succeed while reporting nothing. Two images to build, version and scan instead
   of one.
 
+### 3.4 Cluster-configuration inputs
+
+Once the setup is chosen, the container cluster has to be described. The table
+below is the full input surface, split into what a run cannot start without and
+what only applies under a stated condition. "Mandatory" means the run fails, or
+reports nothing, if the input is absent or wrong.
+
+| Input | Tightly coupled | Loosely coupled | Notes |
+|---|---|---|---|
+| Cluster / kube context hosting the run worker | Mandatory | Mandatory | The launcher's cluster. Loosely coupled may dispatch to a *second* cluster |
+| Namespace for run pods | Mandatory | Mandatory | Supplied by the code location's `DAGSTER_CONTAINER_CONTEXT`; the launcher's `job_namespace` is only a fallback |
+| Code location image reference and tag | Mandatory | Mandatory | Never `latest`; check C3 |
+| Executor selection | Mandatory | Mandatory | Bounds step granularity; loosely coupled it bounds only dispatch concurrency |
+| Service account for run pods | Mandatory | Mandatory | Must have `automountServiceAccountToken: true` |
+| Run-pod egress to the metadata database | **Mandatory** | Not applicable | The pod that writes run events |
+| Namespace for step pods | Conditional — `k8s_job_executor` only | Not applicable | Defaults to the run pod's namespace |
+| RBAC to create and watch Jobs | Conditional — `k8s_job_executor` only | **Mandatory**, in the payload namespace | Same verbs, different namespace |
+| Shared RWX volume plus an I/O manager `base_dir` under it | Conditional — `k8s_job_executor` only | Optional | Without it step pods cannot read each other's outputs |
+| Per-step resource requests and limits | Optional | Optional | Multiply by step or dispatch concurrency |
+| Payload image reference and tag | Not applicable | **Mandatory** | Must share the code location's commit tag |
+| Namespace for payload workloads | Not applicable | **Mandatory** | May be in another cluster or trust boundary |
+| Message channel and its reader | Not applicable | **Mandatory** | Pod log stream by default; RBAC on `pods/log` *is* the channel |
+| Payload wait timeout | Not applicable | Optional | Defaults to 24 h — lower it so scheduling failures surface |
+| Vault role and secret paths | Conditional — if the workflow reads secrets | Conditional — launching pod only | Payloads should receive only `extras` and explicit `env` |
+| Object-storage endpoint and credentials | Conditional — if the workflow uses S3 | Conditional — if the *payload* uses S3 | |
+| Image pull secret | Conditional — non-anonymous registry | Conditional — non-anonymous registry | Applies to the SA that runs the pod, which for dispatched payload Jobs is `default` |
+
+### 3.5 Minimum input set
+
+The list below is the reusable baseline: the smallest set of values a team has to
+agree on before a workflow can be configured against a cluster. Record it once
+per workflow, in the workflow's own repository, and carry it into whatever
+configuration format the team already uses.
+
+**Both setups**
+
+1. `cluster` — the cluster the run worker lands in.
+2. `run_namespace` — namespace for run pods.
+3. `code_location_image` — image reference including an immutable tag.
+4. `executor` — executor name plus its concurrency bound.
+5. `service_account` — the identity run pods assume.
+
+**Tightly coupled adds**
+
+6. `metadata_db` — host and port the run and step pods must reach.
+7. `step_namespace` — only when `k8s_job_executor` is used.
+8. `shared_output_path` — RWX mount path, and the I/O manager `base_dir` pinned
+   under it, again only for `k8s_job_executor`.
+
+**Loosely coupled adds**
+
+6. `payload_image` — image reference sharing the code location's commit tag.
+7. `payload_namespace` — where dispatched workloads are created; may be in
+   another cluster.
+8. `message_channel` — the reader in use, and the RBAC verb that carries it.
+
+This guide deliberately does **not** prescribe a configuration schema or key
+names. The names above are descriptive labels for the values, not a required
+format; the examples in section 3.6 express the same baseline in two different
+shapes, and both are correct.
+
+### 3.6 Worked input examples
+
+The reference configuration in [yaml/](../../yaml) shows the baseline expressed
+in the formats each setup already uses. Neither is normative.
+
+**Tightly coupled** — the inputs land in Helm values and in the job's executor
+config, because that is where the platform chart and Dagster expect them:
+
+```yaml
+# yaml/tightly-coupled/values-run-launcher.yaml  (shape, not a fixed schema)
+dagster:
+  runLauncher:
+    type: K8sRunLauncher
+    config:
+      k8sRunLauncher:
+        jobNamespace: dagster              # run_namespace
+        failPodOnRunFailure: true
+        runK8sConfig:
+          containerConfig:
+            env:                           # metadata_db, S3 and Vault reachability
+              - name: S3_ENDPOINT_URL
+                value: https://s3.dev.simpl-europe.eu
+```
+
+```python
+# step_namespace, shared_output_path and the executor travel with the job
+tightly_coupled_k8s_job = distributed_execution_reference.to_job(
+    executor_def=k8s_job_executor,
+    resource_defs={"io_manager": fs_io_manager.configured({"base_dir": SHARED_IO_BASE_DIR})},
+    config={"execution": {"config": {"step_k8s_config": STEP_K8S_CONFIG}}},
+)
+```
+
+**Loosely coupled** — the same baseline, but the cluster-facing half is the
+payload image and its namespace. Here they are environment values set on both the
+code location and the run launcher, because the dispatching pod is a run pod:
+
+```yaml
+# yaml/loosely-coupled/values-pipes-payload.yaml  (shape, not a fixed schema)
+dagster:
+  dagster-user-deployments:
+    deployments:
+      - name: distributed-execution
+        env:
+          - name: PIPES_PAYLOAD_IMAGE                # payload_image, tag-locked
+            value: <registry>/distributed-execution/payload:0.0.0
+          - name: PIPES_PAYLOAD_NAMESPACE            # payload_namespace
+            value: dagster
+```
+
+```python
+PAYLOAD_IMAGE = os.environ.get("PIPES_PAYLOAD_IMAGE", ...)
+PAYLOAD_NAMESPACE = os.environ.get("PIPES_PAYLOAD_NAMESPACE", "dagster")
+```
+
+The `message_channel` input has no key in either file. The default reader is the
+pod log stream, so it is configured by granting `pods/log` in
+[yaml/loosely-coupled/rbac-pipes-dispatch.yaml](../../yaml/loosely-coupled/rbac-pipes-dispatch.yaml).
+An input can be mandatory and still not be a configuration field.
+
+Note what is absent from the loosely coupled example: no metadata database, no
+Vault role, no object-storage credentials. Those inputs are not optional there —
+they do not exist, which is the operational consequence of the choice.
+
 ---
 
-## 4. AC3 — Readiness checks
+## 4. Readiness checks
 
 Pre-run validation items, their expected evidence, and the symptoms and
 corrections for common misconfigurations are maintained separately, in
@@ -193,7 +314,7 @@ corrections for common misconfigurations are maintained separately, in
 
 ---
 
-## 5. AC4 — Configuring execution-target linkage in code
+## 5. Configuring execution-target linkage in code
 
 ### 5.1 Configuration constructs
 
@@ -233,12 +354,18 @@ tightly_coupled_local_job = distributed_execution_reference.to_job(
 ```python
 tightly_coupled_k8s_job = distributed_execution_reference.to_job(
     name="tightly_coupled_k8s_job",
-    executor_def=k8s_job_executor.configured(
-        {
+    executor_def=k8s_job_executor,
+    resource_defs={
+        "io_manager": fs_io_manager.configured({"base_dir": SHARED_IO_BASE_DIR})
+    },
+    config={
+        "execution": {
+            "config": {
                 "image_pull_policy": "IfNotPresent",
                 "step_k8s_config": STEP_K8S_CONFIG,
             }
-    ),
+        }
+    },
     tags={**COMMON_TAGS, "executor": "k8s_job"},
 )
 ```
@@ -253,6 +380,22 @@ operationally:
   [yaml/tightly-coupled/rbac-step-executor.yaml](../../yaml/tightly-coupled/rbac-step-executor.yaml).
 - Per-step resource requests apply per pod, so the aggregate request for a
   fan-out step is the per-step request multiplied by concurrency.
+- **Step outputs must land on storage every step pod can read.** With the default
+  filesystem I/O manager each pod writes to its own container filesystem, so a
+  downstream step opens a path that does not exist there. `STEP_K8S_CONFIG` mounts
+  the RWX `dagster-shared-pvc` at `/dagster/shared`, and the job pins
+  `fs_io_manager`'s `base_dir` underneath it.
+- Pin `base_dir` directly rather than relying on `DAGSTER_HOME`. The default
+  `base_dir` is `$DAGSTER_HOME/storage`, but the Dagster chart already injects
+  `DAGSTER_HOME=/tmp/dagster` into the step container; a second entry appended by
+  `step_k8s_config` does not displace it, so the override is silently ignored and
+  outputs keep going to `/tmp/dagster/storage`.
+
+The executor's defaults go through `config=` rather than
+`k8s_job_executor.configured(...)`. `.configured()` collapses the executor's
+config schema to `Any`, which makes a run-config `execution:` block accepted and
+then discarded, and makes `job_image` and `job_namespace` impossible to supply at
+launch time.
 
 The evidence that the switch took effect is in `summarise_results` output
 metadata, and the two fields say different things:
@@ -485,6 +628,32 @@ on a cluster it is what distinguishes `k8s_job_executor` from
 holds the namespace the run pod landed in, which is the placement half of the
 execution target.
 
+### 5.7 Dagster UI reference
+
+The screenshots below were captured from the local reference service on
+2026-09-02. They show the surfaces participants use to inspect job linkage and
+run evidence. Cluster placement is evidenced separately by the sandbox run IDs
+in the readiness checklist.
+
+All five reference jobs are registered under one code location:
+
+![Distributed execution job list](images/distributed-execution-job-list.png)
+
+The job overview exposes the graph whose executor or dispatch op defines the
+integration pattern:
+
+![Tightly coupled reference graph](images/distributed-execution-job-graph.png)
+
+The Launchpad shows that `execution_target` and `executor` are job tags persisted
+by code configuration, not a separate runtime-only selector:
+
+![Execution-target tags in Launchpad](images/distributed-execution-launchpad.png)
+
+A successful local run displays the step events and final status used alongside
+output metadata as retained evidence:
+
+![Successful tightly coupled reference run](images/distributed-execution-successful-run.png)
+
 ---
 
 ## 6. Upstream documentation
@@ -506,24 +675,59 @@ execution target.
 
 ## 8. Outstanding work
 
-Tracked under SIMPL-30451.
+Tracked under SIMPL-30451 and SIMPL-30787.
 
-| Item | AC | Status |
+| Item | Status |
+|---|---|
+| Execution-target choices, prerequisites, workflow-level documentation | Complete |
+| Comparison, environment-fit indicators, trade-offs | Complete |
+| Cluster-configuration inputs, mandatory vs optional, minimum input set | Complete |
+| Readiness checklist and evidence mapping | Complete; both Kubernetes paths cluster-verified |
+| Misconfiguration symptoms and corrections | Complete; corrected with observed cluster failure modes |
+| Configuration constructs and persistence | Complete |
+| Before-and-after example, tightly coupled | Complete |
+| Before-and-after example, switching to loosely coupled | Complete |
+| Runnable tightly coupled reference implementations | Complete |
+| Runnable loosely coupled reference implementation | Complete (subprocess and Kubernetes transports both verified) |
+| Both images build; payload image passes the isolation check (L11) | Complete |
+| Locally runnable jobs launched from the Dagster UI, evidence recorded in section 5.6 | Complete |
+| Payload image published to a container registry | Complete on the sandbox Gitea registry; the GitLab registry still pending |
+| GitLab pipeline builds both images | **Pending** — the shared `ds.gitlab-ci.yml` template builds one image from the root Dockerfile |
+| End-to-end run of `loosely_coupled_k8s_job` on a cluster | Complete — sandbox-cat-dat, 2026-08-31, run `cff9b348…`; see the readiness checklist |
+| `distributed-execution` registered as a code location on a platform Dagster | Complete — tightly coupled run `1d8cb167…` launched through the platform webserver |
+| Screenshots of UI surfaces | Complete — job list, graph, Launchpad linkage and successful run captured in section 5.7 |
+| Platform architecture document update | Complete — root `deployment_diagram.md` includes the service and both runtime-information paths |
+
+---
+
+## 9. Requirement traceability
+
+The sections above are organised by subject, not by ticket. Two stories share
+this material; the mapping below is authoritative for both.
+
+### SIMPL-30787 — defining the execution container cluster
+
+| AC | Requirement | Where it is met |
 |---|---|---|
-| Execution-target choices, prerequisites, workflow-level documentation | AC1 | Complete |
-| Comparison, environment-fit indicators, trade-offs | AC2 | Complete |
-| Readiness checklist and evidence mapping | AC3 | Complete; tightly coupled K8s executor rows not cluster-verified |
-| Misconfiguration symptoms and corrections | AC3 | Complete; tightly coupled K8s symptoms not cluster-verified |
-| Configuration constructs and persistence | AC4 | Complete |
-| Before-and-after example, tightly coupled | AC4 | Complete |
-| Before-and-after example, switching to loosely coupled | AC4 | Complete |
-| Runnable tightly coupled reference implementations | AC4 | Complete |
-| Runnable loosely coupled reference implementation | AC4 | Complete (subprocess and Kubernetes transports both verified) |
-| Both images build; payload image passes the isolation check (L11) | Tech details | Complete |
-| Locally runnable jobs launched from the Dagster UI, evidence recorded in section 5.6 | Tech details | Complete |
-| Payload image published to a container registry | Tech details | Complete on the sandbox Gitea registry; the GitLab registry still pending |
-| GitLab pipeline builds both images | Tech details | **Pending** — the shared `ds.gitlab-ci.yml` template builds one image from the root Dockerfile |
-| End-to-end run of `loosely_coupled_k8s_job` on a cluster | Tech details | Complete — sandbox-cat-dat, 2026-08-31, run `cff9b348…`; see the readiness checklist |
-| `distributed-execution` registered as a code location on a platform Dagster | Tech details | **Pending** — blocked on the 1.13.19 vs 1.12.8 control plane skew |
-| Screenshots of UI surfaces | Tech details | **Pending** — needs a deployed platform instance, not a local dev server |
-| Platform architecture document update | Tech details | **Pending** |
+| AC1 | Prerequisites for both setups | Section 2.3 |
+| AC1 | What must be documented before execution | Section 2.4 |
+| AC1 | Where cluster-runtime connectivity is required, where decoupled reporting is expected | Sections 1.2 and 2.3; the asymmetry is drawn in the section 1.2 diagram |
+| AC2 | Mandatory vs optional inputs per setup | Section 3.4 |
+| AC2 | Examples for both setups, no fixed config naming model | Section 3.6, with the closing note in 3.5 |
+| AC2 | Reusable minimum input set | Section 3.5 |
+| AC3 | Checklist separates tightly from loosely coupled checks | [readiness-checklist.md](readiness-checklist.md) sections 2 and 3; section 1 holds the checks common to both |
+| AC3 | Each item mapped to expected evidence | The *Expected evidence* column of every checklist table |
+| AC4 | Configuration constructs attaching the pattern to a workflow | Section 5.1 |
+| AC4 | Worked before-and-after example across setups | Sections 5.2 and 5.3 |
+| AC4 | At least one runnable reference implementation per setup | Section 5.5 — three tightly coupled, two loosely coupled |
+| AC4 | Selection persisted in configuration artifacts, not runtime-only | Section 5.4 |
+| Tech | Diagrams, screenshots, working config files, upstream links | Sections 1.2, 5.7, [yaml/](../../yaml), section 6 |
+
+### SIMPL-30451 — execution targets and integration patterns
+
+| AC | Subject | Where it is met |
+|---|---|---|
+| AC1 | Choosing an execution target | Section 2 |
+| AC2 | Decision support | Sections 3.1 to 3.3 |
+| AC3 | Readiness checks | Section 4 and the readiness checklist |
+| AC4 | Linkage configured in code | Section 5 |

documents/user-guide/readiness-checklist.md

@@ -1,18 +1,20 @@
 # Distributed Execution: Readiness Checklist
 
 Pre-run validation for a workflow's execution target. Companion to the
-[user guide](distributed-execution-guide.md); this document covers AC3.
+[user guide](distributed-execution-guide.md); this document is the validation
+checklist required by SIMPL-30451 AC3 and SIMPL-30787 AC3. Section 1 holds the
+checks common to both setups, section 2 the tightly coupled checks and section 3
+the loosely coupled ones; every item states the evidence that proves it.
 
 Run these checks **before the first execution** of a workflow, and again after any
 change to the run launcher, the executor, the target namespace or the code
 location image.
 
-> **L4–L10 were cleared on sandbox-cat-dat on 2026-08-31.** Checks L1–L3 and L11,
-> which exercise the payload contract, the message-parsing path and image-level
-> isolation, are verified locally. The **tightly coupled** Kubernetes path has
-> been assessed against the same cluster but not executed — C5, C6, T2, T3, T5
-> and T8 are confirmed from the live instance configuration and the deployed code
-> locations. See [section 2.1](#21-sandbox-state-2026-08-31).
+> **Both Kubernetes paths are cluster-verified.** L4–L10 were cleared on
+> sandbox-cat-dat on 2026-08-31. The tightly coupled `k8s_job_executor` path
+> reached `RUN_SUCCESS` through the platform launcher on 2026-09-01. Checks
+> L1–L3 and L11, which exercise the payload contract, message parsing and image
+> isolation, are also verified locally. See [section 2.1](#21-sandbox-state-2026-08-31).
 
 ---
 
@@ -31,7 +33,7 @@ location image.
 
 | # | Check | How to verify | Expected evidence |
 |---|---|---|---|
-| T1 | Run pod reaches the metadata database | Launch `tightly_coupled_in_process_job` | Run reaches `SUCCESS`; `report_execution_target` output metadata lists `DAGSTER_POSTGRES_HOST`, `DAGSTER_POSTGRES_USER` and `DAGSTER_POSTGRES_DB` under `env_vars_present`, and `env_vars_missing` is empty |
+| T1 | Run pod reaches the metadata database | Launch `tightly_coupled_in_process_job` | Run reaches `SUCCESS`; `report_execution_target` metadata lists `DAGSTER_PG_PASSWORD` under `env_vars_present` without exposing its value, and `env_vars_missing` is empty. Successful step startup is the connectivity proof because the step loads run state from Postgres before executing the op |
 | T2 | Vault injection works | Same run; inspect the run pod | `kubectl -n dagster describe pod <run-pod>` shows the `vault-env` init container completed; no `vault:` literal remains in the process environment |
 | T3 | Object storage is reachable | Same run, if the workflow uses S3 | No `EndpointConnectionError` in run logs; `S3_ENDPOINT_URL` present in `env_vars_present` |
 | T4 | Multiprocess fan-out actually fans out | Launch `tightly_coupled_local_job` | Run succeeds; `summarise_results` metadata shows one entry per unit in `contributing_workers` and a single entry in `contributing_hosts` — separate processes, same machine |
@@ -67,7 +69,7 @@ instance rather than the platform's Postgres.
 
 `tightly_coupled_in_process_job` also succeeded, reporting a single worker.
 
-### T1's env-var check is wrong for this platform
+### T1's env-var check and platform contract
 
 The run emitted:
 
@@ -75,19 +77,20 @@ The run emitted:
 > `DAGSTER_POSTGRES_HOST`, `DAGSTER_POSTGRES_USER`, `DAGSTER_POSTGRES_DB`.
 > Expected for a loosely coupled target; a misconfiguration for a tightly coupled one.
 
-That warning would fire on a **correctly configured** platform run pod. The chart
-does not set those variables: it injects `DAGSTER_PG_PASSWORD` from
-`dagster-postgresql-secret` and bakes host, user and database into `dagster.yaml`.
-So `TIGHTLY_COUPLED_ENV_VARS` in `preflight.py` names variables Simpl does not
-use, and the check reports a misconfiguration that is not one. The reachability
-probe above is the part that carries real evidence.
+That warning was a false positive on a **correctly configured** platform run
+pod. The chart does not set those variables: it injects `DAGSTER_PG_PASSWORD`
+from `dagster-postgresql-secret` and stores host, user and database in
+`dagster.yaml`. The reference check now tests for `DAGSTER_PG_PASSWORD` and
+reports only its name, never its value. Database reachability is evidenced by
+successful step startup: a tightly coupled step must load its run state from
+the configured Dagster instance before `report_execution_target` can execute.
 
 ### T6 and T7, executed 2026-08-31
 
 Running them needed three things the earlier attempt lacked: dagster pinned to
 1.12.8 to match the control plane, so writing to the live `dataprovider01_dagster`
-could not trigger a migration; `DAGSTER_HOME` moved onto the RWX
-`dagster-shared-pvc`, because the default I/O manager writes step outputs under
+could not trigger a migration; step outputs moved onto the RWX
+`dagster-shared-pvc`, because the default I/O manager writes them under
 `$DAGSTER_HOME/storage` and each step pod otherwise gets its own empty volume;
 and `automount_service_account_token: true` in `step_k8s_config`, because
 `dagster-svc-account` disables it by default. The probe is
@@ -125,6 +128,39 @@ A run submitted the normal way, through the webserver to `K8sRunLauncher`, gets
 T6/T7 end to end, including `summarise_results`, needs the code location
 registered on the platform Dagster.
 
+### T6 and T7 closed end to end, 2026-09-01
+
+With the code location registered on the platform Dagster,
+`tightly_coupled_k8s_job` was launched from the webserver and reached
+`RUN_SUCCESS` as run `1d8cb167-9fb8-4a34-a036-789eac381e13`. `K8sRunLauncher`
+created `dagster-run-1d8cb167-…` in `dataprovider01`, `K8sStepHandler` created one
+`dagster-step-…` Job per step, and every step — including `summarise_results` —
+completed. No hand-rolled probe, no run-monitoring 403.
+
+**The last thing in the way was where step outputs are written.** Two earlier
+attempts failed on
+`FileNotFoundError: /tmp/dagster/storage/<run-id>/generate_work_units/result/unit_0`,
+with every step pod using its own container filesystem. Mounting
+`dagster-shared-pvc` into the step pods was necessary but not sufficient: setting
+`DAGSTER_HOME` through `step_k8s_config` does not take effect, because the Dagster
+chart already injects `DAGSTER_HOME=/tmp/dagster` and the appended second entry
+does not displace the first. The fix is to bypass `DAGSTER_HOME` and pin the I/O
+manager instead — `fs_io_manager.configured({"base_dir": SHARED_IO_BASE_DIR})` on
+the job, with `SHARED_IO_BASE_DIR` under the mounted PVC. The successful run logs
+confirm it:
+
+> Handled output "result" using IO manager "io_manager" —
+> `/dagster/shared/distributed-execution/storage/1d8cb167-…/generate_work_units/result/unit_0`
+
+The platform run exposed the false-positive `DAGSTER_POSTGRES_*` warning
+described under T1 above. The reference now checks the chart's actual
+`DAGSTER_PG_PASSWORD` injection instead.
+
+> Note: `report_execution_target` reported `namespace: <not-in-kubernetes>` on a
+> step pod that plainly was in Kubernetes. The check reads the service account
+> namespace file, which is absent when the token is not automounted. It is a
+> property of the probe, not of the run.
+
 **C6 passes, contrary to what `job_namespace` alone suggests.** The launcher's
 `job_namespace: dagster` is only a default. Each code location ships a
 `DAGSTER_CONTAINER_CONTEXT` environment variable — generated by the chart — whose
@@ -141,10 +177,8 @@ Both deployed code locations carry it, so both would launch correctly:
 | `template-code-location` | `dataprovider01` |
 | `semaphore-ui` | `dataprovider01` |
 
-What is still true is that **no run has ever executed on this cluster** —
-`dataprovider01` holds no pods labelled `dagster/run-id` and the daemon log shows
-no launch activity. That appears to be because nothing has been launched, not
-because anything is broken.
+The successful run confirms that the code-location container context places run
+and step pods in `dataprovider01` with the required namespace-local dependencies.
 
 The residual risk is narrower than a broken namespace, but real: a code location
 that reaches the launcher *without* a container context namespace inherits
@@ -216,6 +250,7 @@ rather than at the omission.
 | `env_vars_missing` is non-empty in `report_execution_target` metadata | Env vars are set on the code location deployment but not on the run pod | Add them under `runLauncher.config.k8sRunLauncher.runK8sConfig.containerConfig.env` — code location env is **not** inherited by run pods |
 | A literal `vault:...` string appears as a value at runtime | Vault mutating webhook did not process the pod | Verify the `vault.security.banzaicloud.io/*` annotations are on the **run pod** template, not only the code location pod |
 | Steps hang in `STARTING` with `k8s_job_executor` | Service account lacks Job create/watch permission | Apply `yaml/tightly-coupled/rbac-step-executor.yaml` and confirm with `kubectl auth can-i` |
+| `FileNotFoundError` loading a step input from `/tmp/dagster/storage/...` | Step pods each use their own container filesystem for I/O manager output | Mount an RWX volume into step pods via `step_k8s_config` **and** pin `fs_io_manager`'s `base_dir` under it. Setting `DAGSTER_HOME` in `step_k8s_config` does not work — the chart's `DAGSTER_HOME=/tmp/dagster` is injected first and wins |
 | `contributing_hosts` shows one host when `k8s_job_executor` is configured | Run tags or Launchpad config overrode the executor, or the image predates the change | Confirm the code location reloaded after the image bump; check the run's *Config* tab for an `execution:` override |
 | Step pods `OOMKilled` under fan-out | Per-step memory limit applied per pod, aggregate exceeded quota | Raise `step_k8s_config` limits or lower step concurrency; the two multiply |
 | Postgres refuses connections once fan-out grows | Each step pod is an independent DB client | Reduce step concurrency, raise the Postgres connection limit, or move the fan-out step to a loosely coupled target |
@@ -234,8 +269,10 @@ rather than at the omission.
 | Run cancelled in the UI, payload pod keeps running | Cancellation is not propagated to dispatched workloads automatically | `delete_pod_on_completion` handles the normal path; for cancellation, verify orphaned Jobs and add a cleanup sensor |
 
 > **Partly observed.** The pipes rows were exercised on sandbox-cat-dat on
-> 2026-08-31. Rows describing the tightly coupled `k8s_job_executor` still follow
-> from the dagster-k8s API rather than from observation.
+> 2026-08-31. The tightly coupled `k8s_job_executor` rows were closed on
+> 2026-09-01 by run `1d8cb167-9fb8-4a34-a036-789eac381e13`; the remaining rows in
+> the table above still follow from the dagster-k8s API rather than from
+> observation.
 >
 > One symptom the cluster run added, absent from the table above: a service
 > account with `automountServiceAccountToken: false` — which the platform's

src/distributed_execution/preflight.py

@@ -12,9 +12,7 @@ from urllib.parse import urlparse
 
 # Runtime dependencies a tightly coupled execution pod must be able to resolve.
 TIGHTLY_COUPLED_ENV_VARS = (
-    "DAGSTER_POSTGRES_HOST",
-    "DAGSTER_POSTGRES_USER",
-    "DAGSTER_POSTGRES_DB",
+    "DAGSTER_PG_PASSWORD",
 )
 
 

tests/test_loosely_coupled.py

@@ -65,9 +65,7 @@ def test_each_unit_is_dispatched_to_its_own_external_worker():
 
 
 def test_payload_reports_no_orchestration_credentials(monkeypatch):
-    monkeypatch.delenv("DAGSTER_POSTGRES_HOST", raising=False)
-    monkeypatch.delenv("DAGSTER_POSTGRES_USER", raising=False)
-    monkeypatch.delenv("DAGSTER_POSTGRES_DB", raising=False)
+    monkeypatch.delenv("DAGSTER_PG_PASSWORD", raising=False)
 
     result = loosely_coupled_subprocess_job.execute_in_process()
     mapped = result.output_for_node("dispatch_external_work_subprocess")

tests/test_tightly_coupled.py

@@ -81,6 +81,16 @@ def test_env_var_check_reports_missing_names():
     assert result["missing"] == ["DEFINITELY_NOT_SET_12345"]
 
 
+def test_env_var_check_reports_present_names(monkeypatch):
+    monkeypatch.setenv("DAGSTER_PG_PASSWORD", "not-exposed-in-evidence")
+
+    result = check_env_vars(("DAGSTER_PG_PASSWORD",))
+
+    assert result["passed"] is True
+    assert result["present"] == ["DAGSTER_PG_PASSWORD"]
+    assert "not-exposed-in-evidence" not in repr(result)
+
+
 def test_pod_identity_falls_back_outside_kubernetes(monkeypatch):
     monkeypatch.delenv("DAGSTER_K8S_PIPELINE_RUN_NAMESPACE", raising=False)
 

yaml/tightly-coupled/values-run-launcher.yaml

@@ -13,8 +13,8 @@ dagster:
     config:
       k8sRunLauncher:
         # Namespace the run pods land in. Must be a namespace whose NetworkPolicy
-        # permits egress to Postgres, object storage and Vault - see AC1
-        # prerequisites in the user guide.
+        # permits egress to Postgres, object storage and Vault - see the
+        # prerequisites in section 2.3 of the user guide.
         jobNamespace: dagster
 
         # Surfaces step failures as pod failures so kubectl and Dagster agree.