Revert "moved cm-mlops to https://github.com/mlcommons/cm-mlops"

This reverts commit 31a69f2.

cm-mlops/CHANGES.txt

@@ -0,0 +1,122 @@
+20220718:
+ * fixed local_env_keys in get-python3
+ * added new_env_only_keys to meta to specify which env to keep
+ * fixed problem with adding tags from the selected script during caching
+ * added --skip-compile and --skip-run to script (converted to env CM_SKIP_COMPILE and CM_SKIP_RUN)
+
+20220713:
+ * added local_env_keys to meta
+ * added "env" dict to os_info
+
+20220712:
+ * major script refactoring to support cache tags update from deps
+ * fixed version min/max propagations in deps
+ * improvements to support tags from deps
+
+20220708:
+ * various fixes to handle versions (min/max/default)
+ * various fixes to avoid contamination of ENV from other scripts
+
+20220705:
+ * fixes for remembered selections
+ * added --skip-remembered-selections to "cm run script"
+
+20220704:
+ * fixed a bug with searching for scripts with variations
+ * added the possibilty to update deps from pre/post processing
+ * added --extra-cache-tags and --name for "cm run script"
+ * added prototype of selection caching
+
+20220701:
+ * added dummy "cm test script"
+ * added "--env" to "cm show cache" to show env and state
+ * added "cm show cache"
+
+20220629:
+ * added "detect_version_using_script" in script used to detect python packages
+ * major fix to properly support multiple scripts with the same tags, caching, selection, etc
+ * fixed a bug in version comparison (converting string to int)
+ * added recording of "version" to cache meta
+
+20220628:
+ * fixed local_env with deps
+
+20220623:
+ * important update of versions logic
+
+20220621:
+ * added support for --quiet
+ * changed CM_NEED_VERSION to CM_VERSION
+ * added CM_VERSION_MIN, CM_VERSION_MAX
+ * added cm compare_versions utils --version1=... --version2=...
+ * added support to detect min/max/correct versions
+
+20220617
+ * fixed logic to handle variations (-_): https://github.com/mlcommons/ck/issues/243
+
+20220616:
+ * changed "cached" to "cache" automation
+
+20220615:
+ * major update of script (remove parallel env/new_env and state/new_state).
+   keep global env & state and detect changes automatically
+ * major simplification of "script"
+ * removed "installed" to be more understandable
+ * added "cached" to be more understandable
+
+20220609:
+ * added "versions" key to the CM script meta
+   it works similar to "variations" and is forced by --version
+ * changed "ic" to "script" in "experiment" automation
+
+20220608:
+ * updated "variations" logic in "script"!
+   meta['default_variation'] (str): only one of many
+   meta['default_variations'] (list): multiple choices
+
+ * deprecated "ic" automation. Use "script" instead!
+
+20220607:
+ * added strip_folders to utils/unzip_file
+ * fixed minor bugs in CM script
+
+20220606:
+ * added "name" key to deps (list of names and UIDs)
+ * added "add_deps_tags" in variations and in CMD ({"name":"tag(s)"})
+ * added "deps" to variations to be merged with the list of current deps
+ * added --input and --output for cm run script converted to env CM_INPUT and CM_OUTPUT
+   useful to create interactive CM scripts to process files
+
+20220605:
+ * clean tmp files in "script" automation by default and keep them using --dirty flag
+
+20220603:
+ * added "skip" and "deps" to postprocess to call other scripts.
+   For example call install LLVM if detect LLVM fails...
+
+ * added "script" automation to substitute less intuitive "ic"
+
+20220601:
+ * added version, path, skip_install and post_deps to IC 
+ * added --new to IC to detect new components
+
+20220530:
+ * updated ic automation to read tmp-run-state.json 
+   and merge it with the "new_state" dict
+
+20220524:
+ * changed directory ck2-repo-mlops to cm-devops
+
+20220517:
+ * Changed CM_PATH_LIST to +PATH
+ * Added general support for +ENV that is expanded to ENV=val1;val2;...:${ENV}
+
+20220511:
+ * Better handle exceptions in utils.download_file
+ * Added support for variations in intelligent components (ic)
+ * Fixed bugs in IC
+ * Added "_" prefix in tags to specify variation of IC
+ * Record env.sh in "installed artifacts even if bat file is not executed
+ * Fixed experiment directory naming on Windows
+ * Added "cm version ic" (#233)
+

cm-mlops/README.md

@@ -0,0 +1,47 @@
+# CM repository to enable more determinstic, portable and reproducible MLOps
+
+[![CM repository](https://img.shields.io/badge/Collective%20Mind-compatible-blue)](https://github.com/mlcommons/ck/tree/master/cm)
+[![CM artifact](https://img.shields.io/badge/Artifact-automated%20and%20reusable-blue)](https://github.com/mlcommons/ck/tree/master/cm)
+
+
+It is becoming very challenging to co-design, optimize and deploy efficient AI Systems in the real world:
+["MLOps Is a Mess But That's to be Expected"](https://www.mihaileric.com/posts/mlops-is-a-mess).
+
+However, [our experience](https://doi.org/10.5281/zenodo.6475385) 
+suggests that it is possible to [apply DevOps principles to MLOps](https://www.datanami.com/2022/03/30/birds-arent-real-and-neither-is-mlops/)
+if we treat all AI, ML and Systems artifacts including models, data sets, frameworks, libraries and scripts as "code" meta packages 
+with dependencies on other artifacts, operating systems and hardware.
+
+We use this [CM-based repository](https://github.com/mlcommons/ck/tree/master/cm) 
+as a common playground and a common language to learn with the community
+how to make benchmarking, optimization, co-design and deployment
+of complex ML Systems more deterministic, portable and reproducible 
+across continusly changing software and hardware stacks.
+
+
+# How to use
+
+## Install CM toolkit and dependencies
+
+Install the CM toolkit as described [here](https://github.com/mlcommons/ck/blob/master/cm/docs/installation.md).
+
+## Install this CM repository
+
+Use CM to install this repository on your system:
+
+```bash
+$ cm pull repo mlcommons@ck
+```
+
+You can see this and other CM-compatible repositories installed on your system as follows:
+```bash
+$ cm list repo
+```
+
+You can list reusable automations as follows:
+```bash
+$ cm find automation
+```
+
+
+*More to come soon ...*

cm-mlops/automation/cache/_cm.json

@@ -0,0 +1,9 @@
+{
+  "alias": "cache",
+  "automation_alias": "automation",
+  "automation_uid": "bbeb15d8f0a944a4",
+  "tags": [
+    "automation"
+  ],
+  "uid": "541d6f712a6b464e"
+}

cm-mlops/automation/cache/module.py

@@ -0,0 +1,133 @@
+import os
+
+from cmind.automation import Automation
+from cmind import utils
+
+class CAutomation(Automation):
+    """
+    Automation actions
+    """
+
+    ############################################################
+    def __init__(self, cmind, automation_file):
+        super().__init__(cmind, __file__)
+
+    ############################################################
+    def test(self, i):
+        """
+        Test automation
+
+        Args:
+          (CM input dict): 
+
+          (out) (str): if 'con', output to console
+
+          automation (str): automation as CM string object
+
+          parsed_automation (list): prepared in CM CLI or CM access function
+                                    [ (automation alias, automation UID) ] or
+                                    [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]
+
+          (artifact) (str): artifact as CM string object
+
+          (parsed_artifact) (list): prepared in CM CLI or CM access function
+                                    [ (artifact alias, artifact UID) ] or
+                                    [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]
+
+          ...
+
+        Returns:
+          (CM return dict):
+
+          * return (int): return code == 0 if no error and >0 if error
+          * (error) (str): error string if return>0
+
+          * Output from this automation action
+
+        """
+
+        import json
+        print (json.dumps(i, indent=2))
+
+        return {'return':0}
+
+    ############################################################
+    def show(self, i):
+        """
+        Show cache
+
+        Args:
+          (CM input dict): 
+
+          (out) (str): if 'con', output to console
+
+          (env) (bool): if True, show env from cm-cached-state.json
+          
+          ...
+
+        Returns:
+          (CM return dict):
+
+          * return (int): return code == 0 if no error and >0 if error
+          * (error) (str): error string if return>0
+
+          * Output from this automation action
+
+        """
+        import json
+
+        # Check parsed automation
+        if 'parsed_automation' not in i:
+           return {'return':1, 'error':'automation is not specified'}
+
+        console = i.get('out') == 'con'
+
+        show_env = i.get('env', False)
+
+        # Find CM artifact(s)
+        i['out'] = None
+        r = self.search(i)
+
+        if r['return']>0: return r
+
+        lst = r['list']
+        for artifact in lst:
+            path = artifact.path
+            meta = artifact.meta
+            original_meta = artifact.original_meta
+
+            alias = meta.get('alias','')
+            uid = meta.get('uid','')
+
+            tags = sorted(meta.get('tags',[]))
+            version = meta.get('version','')
+
+            if console:
+                print ('')
+                print ('* cache:: {}'.format(uid))
+                print ('    Tags: {}'.format(tags))
+                if version!='':
+                    print ('    Version: {}'.format(version))
+                print ('    Path: {}'.format(path))
+
+            if show_env and console:
+                path_to_cached_state_file = os.path.join(path, 'cm-cached-state.json')
+
+                if os.path.isfile(path_to_cached_state_file):
+                    r =  utils.load_json(file_name = path_to_cached_state_file)
+                    if r['return']>0: return r
+
+                    # Update env and state from cache!
+                    cached_state = r['meta']
+
+                    new_env = cached_state.get('new_env', {})
+                    if len(new_env)>0:
+                        print ('    New env:')
+                        print (json.dumps(new_env, indent=6, sort_keys=True).replace('{','').replace('}',''))
+
+                    new_state = cached_state.get('new_state', {})
+                    if len(new_state)>0:
+                        print ('    New state:')
+                        print (json.dumps(new_env, indent=6, sort_keys=True))
+
+        return {'return':0, 'list': lst}

cm-mlops/automation/docker/_cm.json

@@ -0,0 +1,9 @@
+{
+  "alias": "docker",
+  "automation_alias": "automation",
+  "automation_uid": "bbeb15d8f0a944a4",
+  "tags": [
+    "automation"
+  ],
+  "uid": "2d90be7cab6e4d9f"
+}

cm-mlops/automation/docker/module.py

@@ -0,0 +1,51 @@
+import os
+
+from cmind.automation import Automation
+from cmind import utils
+
+class CAutomation(Automation):
+    """
+    CM "docker" automation actions
+    """
+
+    ############################################################
+    def __init__(self, cmind, automation_file):
+        super().__init__(cmind, __file__)
+
+    ############################################################
+    def test(self, i):
+        """
+        Test automation
+
+        Args:
+          (CM input dict): 
+
+          (out) (str): if 'con', output to console
+
+          automation (str): automation as CM string object
+
+          (artifact) (str): artifact as CM string object
+
+          parsed_automation (list): prepared in CM CLI or CM access function
+                                    [ (automation alias, automation UID) ] or
+                                    [ (automation alias, automation UID), (automation repo alias, automation repo UID) ]
+
+          (parsed_artifact) (list): prepared in CM CLI or CM access function
+                                    [ (artifact alias, artifact UID) ] or
+                                    [ (artifact alias, artifact UID), (artifact repo alias, artifact repo UID) ]
+
+          ...
+
+        Returns:
+          (CM return dict):
+
+          * return (int): return code == 0 if no error and >0 if error
+          * (error) (str): error string if return>0
+
+          * Output from this automation action
+        """
+
+        import json
+        print (json.dumps(i, indent=2))
+
+        return {'return':0}

cm-mlops/automation/experiment/_cm.json

@@ -0,0 +1,9 @@
+{
+  "alias": "experiment",
+  "automation_alias": "automation",
+  "automation_uid": "bbeb15d8f0a944a4",
+  "tags": [
+    "automation"
+  ],
+  "uid": "a0a2d123ef064bcb"
+}