`canari_ml preprocess` CLI Help¶

Run the following command to get the help information for canari_ml preprocess command:

$ canari_ml preprocess --help

usage: canari_ml preprocess [-h] {train,predict} ...

positional arguments:
  {train,predict}

options:
  -h, --help       show this help message and exit

Run the following command to get the help information for canari_ml preprocess train command:

$ canari_ml preprocess train --help

train_56480d80 is powered by Hydra.

== Configuration groups ==
Compose your configuration from those groups (group=option)

callbacks: default, early_stopping, model_checkpoint
common: default
hydra_config: predict, train
logger: csv, tensorboard, wandb
model: default, unet
paths: default, download, plot, postprocess, predict, preprocess, train
plot: default, ua700
postprocess: default, netcdf, plot_ua700
predict: default
preprocess: default
profiler: pytorch
train: default
trainer: default


== Config ==
Override anything in the config (foo.bar=value)

input:
  name: ''
  forecast_length: 3
  lag_length: ${input.forecast_length}
  vars:
    absolute:
    - sic
    - tas
    - tos
    - ua2
    - ua10
    - ua50
    - ua100
    - ua250
    - ua500
    - ua700
    anomaly:
    - zg2
    - zg10
    - zg50
    - zg100
    - zg250
    - zg500
    - zg700
  dates:
    train:
      start:
      - 2000-1-5
      end:
      - 2000-1-5
    val:
      start:
      - 2000-4-5
      end:
      - 2000-4-5
    test:
      start:
      - 2000-2-1
      end:
      - 2000-2-1
    predict:
      start:
      - 2000-2-1
      end:
      - 2000-2-1
reproject:
  source_crs: EPSG:4326
  target_crs: EPSG:6931
  shape: 500
preprocess_era5:
  implementation: canari_ml.data.processors.cds:ERA5PreProcessor
  smooth_sigma: 0.5
preprocess_mask:
  implementation: canari_ml.data.masks.era5:Masks
  channel_name: hemisphere
preprocess_cache:
  implementation: serial
  output_batch_size: 4
workers: 16
frequency: DAY
hemisphere: north
params:
  config_suffix: ${frequency}.${hemisphere}
  config_name: ${hydra:job.name}
hashes:
  reproject: ${compute_step_hash:[${input}, ${preprocess_type}, ${reproject}], ${input.name}}
  preprocess_era5: ${compute_step_hash:[${hashes.reproject}, ${preprocess_era5}],
    ${input.name}}
  preprocess_mask: ${compute_step_hash:[${hashes.preprocess_era5}, ${preprocess_mask}],
    ${input.name}}
  cache: ${compute_step_hash:[${hashes.preprocess_mask}, ${preprocess_cache}], ${input.name}}
  combined: ${compute_step_hash:[${hashes.preprocess_mask}], ${input.name}}
preprocess_type: train
source_dataset_id: era5
paths:
  download:
    source_data_dir: ${hydra:runtime.cwd}/data
    config_file: ${paths.download.source_data_dir}/data.aws.${frequency}.${hemisphere}.json
  preprocess:
    data_root: ${getcwd:}/preprocessed_data
    preprocessed_data_dir: ${paths.preprocess.data_root}/preprocessed
    reprojected_data_dir: ${paths.preprocess.preprocessed_data_dir}
    normalised_data_dir: ${paths.preprocess.preprocessed_data_dir}
    cache_dir: ${paths.preprocess.data_root}/cache
    loader_file: ${paths.preprocess.preprocessed_data_dir}/loader.${params.config_name}.json
  reproject:
    destination_path: ${paths.preprocess.reprojected_data_dir}/01_reproject${opt_underscore:${input.name}}${opt_underscore:${hashes.reproject}}
    config_file: ${paths.reproject.destination_path}/reproject.${params.config_suffix}.json
  preprocess_era5:
    destination_path: ${paths.preprocess.normalised_data_dir}/02_normalised${opt_underscore:${input.name}}${opt_underscore:${hashes.preprocess_era5}}
    config_file: ${paths.preprocess_era5.destination_path}/processed_era5.${params.config_suffix}.json
  mask:
    destination_path: ${paths.preprocess_era5.destination_path}/mask${opt_underscore:${input.name}}${opt_underscore:${hashes.preprocess_mask}}
    mask_dataset_config_path: ${paths.mask.destination_path}/dataset_config.masks.${params.config_suffix}.json
    mask_config_path: ${paths.mask.destination_path}/processed_era5.masks.${params.config_suffix}.json
  cache:
    destination_path: ${paths.preprocess.cache_dir}/03_cache${opt_underscore:${input.name}}${opt_underscore:${hashes.cache}}
    config_path: ${paths.cache.destination_path}/cached.${params.config_suffix}.json
  train: outputs/${train.name}/training/


Powered by Hydra (https://hydra.cc)
Use --hydra-help to view Hydra specific help

canari_ml preprocess CLI Help¶

`canari_ml preprocess` CLI Help¶