Files
distributed-execution/CHANGELOG.md
ILay aa49420ca6 [SIMPL-30787] Document container-cluster configuration inputs
Add the cluster-configuration input reference the guide was missing: which inputs are mandatory, conditional or optional in each setup, a reusable minimum input set, and worked examples expressed in the format each setup already uses. No configuration schema or key naming is prescribed.

Retitle the subject sections so they no longer carry one story's AC numbering, and add a traceability section mapping SIMPL-30787 and SIMPL-30451 acceptance criteria onto them.

Changelog: added
2026-09-02 18:48:01 +02:00

1.9 KiB

0.1.0 (unreleased)

added

  • Repository skeleton for the distributed-execution supporting data service.
  • User guide covering execution-target selection, decision support and readiness checks.
  • Cluster-configuration input reference: mandatory versus optional inputs per setup, a reusable minimum input set and worked examples in both setups' own formats.
  • Requirement traceability mapping the guide's sections onto SIMPL-30451 and SIMPL-30787.
  • Runnable tightly coupled reference implementations (in_process, multiprocess, k8s_job_executor).
  • Runnable loosely coupled reference implementations via dagster-pipes (PipesSubprocessClient, PipesK8sClient) with a standalone external payload image.
  • Working example configuration and RBAC for both execution targets.

verified

  • tightly_coupled_k8s_job executed end to end on sandbox-cat-dat through the platform K8sRunLauncher (run 1d8cb167-9fb8-4a34-a036-789eac381e13), closing readiness rows T6 and T7. Step outputs are written to the shared dagster-shared-pvc by pinning fs_io_manager's base_dir; DAGSTER_HOME set through step_k8s_config is ignored, because the Dagster chart injects DAGSTER_HOME=/tmp/dagster ahead of it.
  • loosely_coupled_k8s_job reached RUN_SUCCESS on sandbox-cat-dat through the standalone probe (run cff9b348-bfc3-4ac1-ab51-a94892b8e3a0), with four external payload pods reporting over the pod-log message channel.
  • Tightly coupled preflight evidence now checks the chart's actual DAGSTER_PG_PASSWORD injection without exposing its value.
  • Dagster UI screenshots cover the registered jobs, graph, code-configured execution-target tags and a successful reference run.
  • The platform deployment diagram includes the service and both integration patterns.

pending

  • Extend the project-level GitLab pipeline to publish the payload image as well as the code-location image.