j.r/distributed-execution
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[SIMPL-30451] Add a standalone cluster probe for the pipes transport

Browse Source 
Runs loosely_coupled_k8s_job from one throwaway pod with only the RBAC the pipes client needs, clearing readiness checks L4-L9 without deploying a code location, a webserver or a database. Because the pod is its own control plane it also sidesteps the 1.13.19 vs 1.12.8 skew that blocks registering this service against the sandbox Dagster.

Verified in the container beforehand: the CLI form reaches RUN_SUCCESS, the multiprocess executor fans out to four dynamic steps, and PipesK8sClient selects in-cluster credentials from KUBERNETES_SERVICE_HOST, so no code change is needed to run it in a pod.

Changelog: added
This commit is contained in:
ILay
2026-08-31 17:46:37 +02:00
parent b47c530d46
commit 515eef7736
3 changed files with 176 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
documents/user-guide/readiness-checklist.md
View File

@@ -58,6 +58,11 @@ location image.
Checks L4–L9 require a cluster. L1–L3, L10 and L11 run on a laptop and should
gate every change to the payload or the dispatching op.
`yaml/loosely-coupled/probe-pipes-k8s.yaml` clears L4–L9 in a single run from a
throwaway namespace, without deploying a code location or a Dagster control
plane. Prefer it over assembling the cluster checks by hand: the RBAC it grants
is exactly the set L5 and L6 ask about, so a failure localises immediately.
---
## 4. Common misconfiguration symptoms