HEAL Data Dictionaries Converter

The script scripts/convert2vlmd.py organises HEAL data dictionary files into the directory structure expected by the HEAL Data Platform. It retrieves project metadata from the HEAL metadata service, creates the required directory structure, and processes each data dictionary file.

Modes

Extract mode

Provide --clean_study_directory only. Each CSV or Stata file is converted to VLMD format using healdata-utils, the original file is copied to input/, and a metadata.yaml is generated.

Copy mode

Provide both --clean_study_directory and --vlmd_directory. Use this when VLMD files have already been generated externally (e.g. converted from a CDISC Define-XML). Each VLMD file is validated, copied to vlmd/, the corresponding original input file is copied to input/, and a metadata.yaml is generated. No extraction is performed.

Output directory structure

<output_directory>/
└── data-dictionaries/
    └── <hdp_id>/               (or <project> if --project is supplied)
        ├── input/
        │   └── <original files>
        └── vlmd/
            └── <dd_name>/
                ├── <hdp_id>_<dd_name>.json   (or .csv in copy mode)
                └── metadata.yaml

Command-Line Arguments

Argument	Type	Required	Description
--clean_study_directory	str	Yes	Directory containing original input files (CSV/Stata/XML etc.). In extract mode these are converted to VLMD; in copy mode they are copied to input/.
--vlmd_directory	str	No	Directory containing pre-built VLMD files (.json or .csv). When provided, activates copy mode. Requires --clean_study_directory.
--output_directory	str	Yes	Target directory (ideally a clone of heal-data-dictionaries). Outputs are written to <output_directory>/data-dictionaries/<hdp_id>/.
--hdp_id	str	Yes	HEAL project identifier (e.g. HDP01341). Used to query MDS and name the output directory.
--appl_id	str	No	Award number. Auto-resolved from MDS if omitted.
--project	str	No	Override the output subdirectory name (defaults to hdp_id).
--project_type	str	No	Study type string written to metadata (default: "Research Programs").
--overwrite	flag	No	Re-process and overwrite outputs even if they already exist.

Usage Examples

Extract mode — convert CSV data dictionaries to VLMD:

./scripts/convert2vlmd.py \
  --clean_study_directory <path-to-input-files> \
  --output_directory <path-to-heal-data-dictionaries> \
  --hdp_id HDP12345 \
  --project_type "Research Programs"

Copy mode — validate and copy pre-built VLMD files:

./scripts/convert2vlmd.py \
  --clean_study_directory <path-to-original-input-files> \
  --vlmd_directory <path-to-vlmd-files> \
  --output_directory <path-to-heal-data-dictionaries> \
  --hdp_id HDP12345 \
  --project_type "Research Programs"

With explicit project directory and APPL ID:

./scripts/convert2vlmd.py \
  --clean_study_directory <path-to-input-files> \
  --output_directory <path-to-heal-data-dictionaries> \
  --hdp_id HDP12345 \
  --appl_id 9877133 \
  --project BESTTrial \
  --project_type "HEAL Study"

Outputs will be written to <output_directory>/data-dictionaries/BESTTrial/.

Dependencies

• Python 3.x
• requests
• pyyaml
• click
• pandas
• heal package (provides vlmd_extract, vlmd_validate)

Install dependencies with:

pip install -r requirements.txt

License

See repository root for license information.

Contact

For more information or support, please contact the HEAL Data Stewards.