From 49f3afd6abbd7d60ac23654402d2d01f574af42e Mon Sep 17 00:00:00 2001 From: ILay Date: Fri, 24 Apr 2026 18:42:44 +0200 Subject: [PATCH] docs(SIMPL-24642): update Development Guide to reflect consolidated structure --- documents/Development Guide.md | 39 ++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 11 deletions(-) diff --git a/documents/Development Guide.md b/documents/Development Guide.md index 0f140ad..23c60d7 100644 --- a/documents/Development Guide.md +++ b/documents/Development Guide.md @@ -9,18 +9,35 @@ By following a *code-first approach*, developers ensure consistency, traceabilit Development must always begin in a local environment. This allows developers to rapidly iterate, test business logic, and validate DAG (Directed Acyclic Graph) structures without impacting production data. ### 2.1 Project Layout -To ensure compatibility with the Simpl-Open platform, every Dagster code location must adhere to the following directory structure: +This repository (`template-code-location`) serves as the **single consolidated code location** for all data services workflows. It contains the jobs, ops, and configurations previously spread across `data-processing`, `dataframe-level-anonymisation`, and `field-level-pseudo-anonymisation`. + ```text -project-root/ -├── dagster_code_location/ -│ ├── jobs/ # Executable workflows -│ ├── ops/ # Individual functional units (business logic) -│ ├── resources/ # External connections (Object storage, APIs, etc...) -│ └── repository.py # Central entry point for the code location -├── tests/ # Unit and integration tests -├── Dockerfile # Containerization instructions -├── pyproject.toml # Dependency management (Poetry/Pip/UV) -└── README.md # Documentation +template-code-location/ +├── src/ +│ └── template_code_location/ +│ ├── repository.py # Unified entry point (all jobs/sensors/resources) +│ ├── data_processing/ # Data cleaning & transformation ops/jobs +│ │ ├── config_models/ +│ │ ├── jobs.py +│ │ └── ops.py +│ ├── dataframe_level_anonymisation/ # k-anonymity, l-diversity, t-closeness +│ │ ├── config_models/ +│ │ ├── jobs.py +│ │ ├── ops.py +│ │ └── utils.py +│ ├── field_level_pseudo_anonymisation/ # Field-level encryption/hashing/redaction +│ │ ├── config_models/ +│ │ ├── techniques/ +│ │ ├── jobs.py +│ │ ├── ops.py +│ │ ├── unstructured_ops.py +│ │ └── utils.py +│ ├── jobs/ # Template example jobs +│ └── ops/ # Template example ops +├── tests/ # All tests (migrated from source repos) +├── Dockerfile +├── pyproject.toml +└── README.md ``` ### 2.2 Code Examples (Ops, Jobs, and Definitions)