docs(SIMPL-24642): update Development Guide to reflect consolidated structure

This commit is contained in:
ILay
2026-04-24 18:42:44 +02:00
parent d14b2dfac4
commit 49f3afd6ab

View File

@@ -9,18 +9,35 @@ By following a *code-first approach*, developers ensure consistency, traceabilit
Development must always begin in a local environment. This allows developers to rapidly iterate, test business logic, and validate DAG (Directed Acyclic Graph) structures without impacting production data.
### 2.1 Project Layout
To ensure compatibility with the Simpl-Open platform, every Dagster code location must adhere to the following directory structure:
This repository (`template-code-location`) serves as the **single consolidated code location** for all data services workflows. It contains the jobs, ops, and configurations previously spread across `data-processing`, `dataframe-level-anonymisation`, and `field-level-pseudo-anonymisation`.
```text
project-root/
├── dagster_code_location/
── jobs/ # Executable workflows
├── ops/ # Individual functional units (business logic)
├── resources/ # External connections (Object storage, APIs, etc...)
└── repository.py # Central entry point for the code location
├── tests/ # Unit and integration tests
├── Dockerfile # Containerization instructions
├── pyproject.toml # Dependency management (Poetry/Pip/UV)
└── README.md # Documentation
template-code-location/
├── src/
── template_code_location/
├── repository.py # Unified entry point (all jobs/sensors/resources)
├── data_processing/ # Data cleaning & transformation ops/jobs
│ ├── config_models/
├── jobs.py
└── ops.py
│ ├── dataframe_level_anonymisation/ # k-anonymity, l-diversity, t-closeness
├── config_models/
│ │ ├── jobs.py
│ │ ├── ops.py
│ │ └── utils.py
│ ├── field_level_pseudo_anonymisation/ # Field-level encryption/hashing/redaction
│ │ ├── config_models/
│ │ ├── techniques/
│ │ ├── jobs.py
│ │ ├── ops.py
│ │ ├── unstructured_ops.py
│ │ └── utils.py
│ ├── jobs/ # Template example jobs
│ └── ops/ # Template example ops
├── tests/ # All tests (migrated from source repos)
├── Dockerfile
├── pyproject.toml
└── README.md
```
### 2.2 Code Examples (Ops, Jobs, and Definitions)