[SIMPL-30451] Fan out work units so the execution target is observable

The reference graph processed every work unit inside a single op, so exactly one worker was ever reported regardless of executor. That made the guide's claim that k8s_job_executor yields several distinct contributing_hosts false, and left the reference implementations unable to demonstrate the executor choice at all. generate_work_units is now a DynamicOut and both graphs map over it, so one step is created per unit and the loosely coupled pattern dispatches one external workload per unit.

Evidence is split into contributing_workers (host and pid, differs per process) and contributing_hosts (differs only across machines), because the previous single field could not distinguish multiprocess fan-out from no fan-out. Tests now assert the mapped step keys rather than a host count, since execute_in_process ignores executor_def and cannot prove executor behaviour on its own. Adds a Windows note: multiprocess_executor did not complete during authoring and left orphaned processes.

Changelog: fixed
This commit is contained in:
ILay
2026-08-26 18:50:01 +02:00
parent 43dbb81a95
commit ccc2e94c2a
11 changed files with 148 additions and 65 deletions

View File

@@ -42,8 +42,16 @@ def test_in_process_job_runs_end_to_end():
summary = result.output_for_node("summarise_results")
assert summary["units"] == 4
assert summary["total"] == 14 # 0 + 1 + 4 + 9
# In-process execution never fans out beyond the run worker.
assert len(summary["contributing_hosts"]) == 1
# In-process execution keeps every step in the run worker's own process.
assert len(summary["contributing_workers"]) == 1
def test_work_units_fan_out_into_one_step_each():
result = tightly_coupled_in_process_job.execute_in_process()
mapped = result.output_for_node("process_work_unit")
# Without this the executor choice would be meaningless: one step cannot span pods.
assert set(mapped) == {"unit_0", "unit_1", "unit_2", "unit_3"}
def test_env_var_check_reports_missing_names():