[SIMPL-30451] Add distributed-execution service with guide and reference implementations
Canonical location for documentation, example workflows and reference service implementations covering distributed execution patterns. Covers AC1-AC4: execution-target selection, decision support, readiness checks and code-level linkage. Tightly coupled jobs and the loosely coupled subprocess transport are verified by the test suite; the Kubernetes pipes transport is implemented but not yet cluster-run and is marked as such in the guide. Changelog: added
This commit is contained in:
54
yaml/values-dagster-distributed-execution.yaml
Normal file
54
yaml/values-dagster-distributed-execution.yaml
Normal file
@@ -0,0 +1,54 @@
|
||||
dagster:
|
||||
# ===========================================================================
|
||||
# CODE LOCATION - distributed-execution
|
||||
# ===========================================================================
|
||||
# Deploys this repository as a Dagster code location so the reference jobs
|
||||
# appear in the UI alongside production workflows.
|
||||
# MANAGED BY GITLAB CI/CD - DO NOT CHANGE MANUALLY
|
||||
dagster-user-deployments:
|
||||
deployments:
|
||||
- name: distributed-execution
|
||||
location_name: distributed-execution
|
||||
enabled: true
|
||||
enableSubchart: true
|
||||
|
||||
codeServerArgs:
|
||||
- "--host"
|
||||
- "0.0.0.0"
|
||||
- "--port"
|
||||
- "4000"
|
||||
- "--python-file"
|
||||
- "src/distributed_execution/repository.py"
|
||||
|
||||
envFrom:
|
||||
- secretRef:
|
||||
name: distributed-execution-dagster-db-secret
|
||||
- configMapRef:
|
||||
name: distributed-execution-dagster-db-config
|
||||
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
repository: code.europa.eu:4567/simpl/simpl-open/data/supporting-data-services/distributed-execution
|
||||
tag: 0.0.0 # DO NOT CHANGE - replaced automatically by the pipeline before releasing
|
||||
|
||||
port: 4000
|
||||
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
|
||||
livenessProbe:
|
||||
enabled: true
|
||||
periodSeconds: 30
|
||||
timeoutSeconds: 5
|
||||
failureThreshold: 3
|
||||
|
||||
readinessProbe:
|
||||
enabled: true
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 3
|
||||
failureThreshold: 3
|
||||
Reference in New Issue
Block a user