Click here to see the table of contents.

• Description
• Information
• Usage
  • CM installation
  • CM script automation help
  • CM CLI
  • CM Python API
  • CM GUI
  • CM modular Docker container
• Customization
  • Variations
  • Script flags mapped to environment
  • Default environment
• Script workflow, dependencies and native scripts
• Script output
• New environment keys (filter)
• New environment keys auto-detected from customize
• Maintainers

Note that this README is automatically generated - don't edit! Use README-extra.md to add more info.

Description

Information

• CM GitHub repository: mlcommons@ck
• GitHub directory for this script: GitHub
• CM meta description for this script: _cm.json
• CM "database" tags to find this script: download,file,download-file
• Output cached?: False

---

Usage

CM installation

Guide

CM pull repository

cm pull repo mlcommons@ck

CM script automation help

cm run script --help

CM CLI

1. cm run script --tags=download,file,download-file[,variations] [--input_flags]

2. cm run script "download file download-file[,variations]" [--input_flags]

3. cm run script 9cdc8dc41aae437e [--input_flags]

• variations can be seen here

• input_flags can be seen here

CM Python API

Click here to expand this section.

import cmind

r = cmind.access({'action':'run'
                  'automation':'script',
                  'tags':'download,file,download-file'
                  'out':'con',
                  ...
                  (other input keys for this script)
                  ...
                 })

if r['return']>0:
    print (r['error'])

CM GUI

cm run script --tags=gui --script="download,file,download-file"

Use this online GUI to generate CM CMD.

CM modular Docker container

TBD

---

Customization

Variations

• No group (any variation can be selected)

  Click here to expand this section.

  • _url.#
    • Environment variables:
      • CM_DOWNLOAD_URL: #
    • Workflow:

• Group "download-tool"

  Click here to expand this section.

  • _cmutil (default)
    • Environment variables:
      • CM_DOWNLOAD_TOOL: cmutil
    • Workflow:
  • _curl
    • Environment variables:
      • CM_DOWNLOAD_TOOL: curl
    • Workflow:
  • _gdown
    • Environment variables:
      • CM_DOWNLOAD_TOOL: gdown
    • Workflow:
      1. Read "deps" on other CM scripts
         • get,generic-python-lib,_package.gdown
           • CM script: get-generic-python-lib
  • _wget
    • Environment variables:
      • CM_DOWNLOAD_TOOL: wget
    • Workflow:

Default variations

_cmutil

Script flags mapped to environment

Click here to expand this section.

• --download_path=value → CM_DOWNLOAD_PATH=value
• --url=value → CM_DOWNLOAD_URL=value

Above CLI flags can be used in the Python CM API as follows:

r=cm.access({... , "download_path":...}

Default environment

Click here to expand this section.

These keys can be updated via --env.KEY=VALUE or env dictionary in @input.json or using script flags.

---

Script workflow, dependencies and native scripts

Click here to expand this section.

1. Read "deps" on other CM scripts from meta
2. Run "preprocess" function from customize.py
3. Read "prehook_deps" on other CM scripts from meta
4. Run native script if exists
   • run.sh
5. Read "posthook_deps" on other CM scripts from meta
6. Run "postrocess" function from customize.py
7. Read "post_deps" on other CM scripts from meta

---

Script output

New environment keys (filter)

• <<<CM_DOWNLOAD_FINAL_ENV_NAME>>>
• CM_DOWNLOAD_DOWNLOADED_PATH
• CM_GET_DEPENDENT_CACHED_PATH

New environment keys auto-detected from customize

• CM_DOWNLOAD_DOWNLOADED_PATH
• CM_GET_DEPENDENT_CACHED_PATH

---

Maintainers

• Open MLCommons taskforce on automation and reproducibility