6532d640f9
The GitLab project lives at simpl-open/data/supporting-data-services/distributed-execution/distributed-execution, so CI_REGISTRY_IMAGE ends in the project name twice. The pipeline pushes to CI_REGISTRY_IMAGE and nowhere else, so the earlier one-segment path named an image nobody builds - the same mismatch that field-level-pseudo-anonymisation and dataframe-level-anonymisation carry today. Changelog: fixed
48 lines
2.3 KiB
YAML
48 lines
2.3 KiB
YAML
# Loosely coupled execution target - code location overrides.
|
|
#
|
|
# Merge into the dagster-user-deployments entry for this code location. These
|
|
# values point the pipes client at the payload image and target namespace.
|
|
#
|
|
# Contrast with yaml/tightly-coupled/values-run-launcher.yaml: there, every
|
|
# execution pod needed S3 and Vault credentials. Here the payload needs none -
|
|
# only the dispatching pod does.
|
|
#
|
|
# VERSION LOCK. The payload tag below must equal the code location image tag in
|
|
# yaml/values-dagster-distributed-execution.yaml. The two images share a commit:
|
|
# the dispatching op and payload/work.py agree on the `units` extras key and on
|
|
# the custom-message shape, and nothing at runtime checks that agreement. A
|
|
# mismatched pair surfaces as "no pipes messages received", which reads like a
|
|
# broken message channel rather than a version skew.
|
|
#
|
|
# For values that work today, see yaml/sandbox/values-sandbox-gitea.yaml. The
|
|
# shared GitLab pipeline (ds.gitlab-ci.yml) builds exactly one image per project -
|
|
# $CI_REGISTRY_IMAGE from the root Dockerfile - so the payload reference below
|
|
# resolves only once that pipeline learns to build a second image. The Gitea
|
|
# workflow in .gitea/workflows/docker-publish.yml already builds both.
|
|
|
|
dagster:
|
|
dagster-user-deployments:
|
|
deployments:
|
|
- name: distributed-execution
|
|
envSecrets: []
|
|
env:
|
|
- name: PIPES_PAYLOAD_IMAGE
|
|
# DO NOT CHANGE the tag - replaced automatically by the pipeline before releasing
|
|
value: code.europa.eu:4567/simpl/simpl-open/data/supporting-data-services/distributed-execution/distributed-execution/payload:0.0.0
|
|
- name: PIPES_PAYLOAD_NAMESPACE
|
|
value: dagster
|
|
|
|
# The dispatching RUN pod creates the payload Job, so the image reference and
|
|
# namespace must also be visible to run pods, not just the code server.
|
|
runLauncher:
|
|
config:
|
|
k8sRunLauncher:
|
|
runK8sConfig:
|
|
containerConfig:
|
|
env:
|
|
- name: PIPES_PAYLOAD_IMAGE
|
|
# DO NOT CHANGE the tag - replaced automatically by the pipeline before releasing
|
|
value: code.europa.eu:4567/simpl/simpl-open/data/supporting-data-services/distributed-execution/distributed-execution/payload:0.0.0
|
|
- name: PIPES_PAYLOAD_NAMESPACE
|
|
value: dagster
|