deploy: 4b54777

genindex.html

@@ -131,6 +131,10 @@ <h2 id="B">B</h2>
 </li>
       </ul></li>
   </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmClient.bring_listener_uptodate">bring_listener_uptodate() (biomero.slurm_client.SlurmClient method)</a>
+</li>
+  </ul></td>
 </tr></table>
 
 <h2 id="C">C</h2>
@@ -197,10 +201,12 @@ <h2 id="G">G</h2>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.get_image_versions_and_data_files">get_image_versions_and_data_files() (biomero.slurm_client.SlurmClient method)</a>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.get_job_status_command">get_job_status_command() (biomero.slurm_client.SlurmClient method)</a>
+</li>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmClient.get_jobs_info_command">get_jobs_info_command() (biomero.slurm_client.SlurmClient method)</a>
 </li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
-      <li><a href="biomero.html#biomero.slurm_client.SlurmClient.get_jobs_info_command">get_jobs_info_command() (biomero.slurm_client.SlurmClient method)</a>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmClient.get_listeners">get_listeners() (biomero.slurm_client.SlurmClient method)</a>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.get_logfile_from_slurm">get_logfile_from_slurm() (biomero.slurm_client.SlurmClient method)</a>
 </li>
@@ -225,6 +231,10 @@ <h2 id="I">I</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.init_workflows">init_workflows() (biomero.slurm_client.SlurmClient method)</a>
+</li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmClient.initialize_analytics_system">initialize_analytics_system() (biomero.slurm_client.SlurmClient method)</a>
 </li>
   </ul></td>
 </tr></table>
@@ -285,6 +295,8 @@ <h2 id="P">P</h2>
 </li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmJob.progress">progress (biomero.slurm_client.SlurmJob attribute)</a>
+</li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.pull_descriptor_from_github">pull_descriptor_from_github() (biomero.slurm_client.SlurmClient method)</a>
 </li>
   </ul></td>
@@ -318,6 +330,8 @@ <h2 id="S">S</h2>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.setup_directories">setup_directories() (biomero.slurm_client.SlurmClient method)</a>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.setup_job_scripts">setup_job_scripts() (biomero.slurm_client.SlurmClient method)</a>
+</li>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmClient.setup_listeners">setup_listeners() (biomero.slurm_client.SlurmClient method)</a>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.setup_slurm">setup_slurm() (biomero.slurm_client.SlurmClient method)</a>
 </li>
@@ -327,13 +341,17 @@ <h2 id="S">S</h2>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.slurm_images_path">slurm_images_path (biomero.slurm_client.SlurmClient attribute)</a>
 </li>
-  </ul></td>
-  <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.slurm_model_images">slurm_model_images (biomero.slurm_client.SlurmClient attribute)</a>
 </li>
+  </ul></td>
+  <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.slurm_model_paths">slurm_model_paths (biomero.slurm_client.SlurmClient attribute)</a>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.slurm_model_repos">slurm_model_repos (biomero.slurm_client.SlurmClient attribute)</a>
+</li>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmJob.SLURM_POLLING_INTERVAL">SLURM_POLLING_INTERVAL (biomero.slurm_client.SlurmJob attribute)</a>
+</li>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmJob.slurm_polling_interval">slurm_polling_interval (biomero.slurm_client.SlurmJob attribute)</a>
 </li>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.slurm_script_path">slurm_script_path (biomero.slurm_client.SlurmClient attribute)</a>
 </li>
@@ -352,6 +370,10 @@ <h2 id="S">S</h2>
 
 <h2 id="T">T</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmJob.task_id">task_id (biomero.slurm_client.SlurmJob attribute)</a>
+</li>
+  </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="biomero.html#biomero.slurm_client.SlurmClient.transfer_data">transfer_data() (biomero.slurm_client.SlurmClient method)</a>
 </li>
@@ -382,6 +404,8 @@ <h2 id="W">W</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="biomero.html#biomero.slurm_client.SlurmJob.wait_for_completion">wait_for_completion() (biomero.slurm_client.SlurmJob method)</a>
+</li>
+      <li><a href="biomero.html#biomero.slurm_client.SlurmJob.wf_id">wf_id (biomero.slurm_client.SlurmJob attribute)</a>
 </li>
   </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>

readme_link.html

@@ -127,7 +127,7 @@
 
   <div class="section" id="biomero-bioimage-analysis-in-omero">
 <h1>BIOMERO - BioImage analysis in OMERO<a class="headerlink" href="#biomero-bioimage-analysis-in-omero" title="Permalink to this headline"></a></h1>
-<p><a class="reference external" href="https://opensource.org/licenses/Apache-2.0"><img alt="License" src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" /></a> <a class="reference external" href="https://zenodo.org/badge/latestdoi/638954891"><img alt="DOI" src="https://zenodo.org/badge/638954891.svg" /></a> <a class="reference external" href="https://pypi.org/project/biomero/"><img alt="PyPI - Version" src="https://img.shields.io/pypi/v/biomero" /></a> <a class="reference external" href="https://pypi.org/project/biomero/"><img alt="PyPI - Python Versions" src="https://img.shields.io/pypi/pyversions/biomero" /></a> <img alt="Slurm" src="https://img.shields.io/badge/Slurm-21.08.6-blue.svg" /> <img alt="OMERO" src="https://img.shields.io/badge/OMERO-5.6.8-blue.svg" /> <a class="reference external" href="https://fair-software.eu"><img alt="fair-software.eu" src="https://img.shields.io/badge/fair--software.eu-%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F-green" /></a> <a class="reference external" href="https://bestpractices.coreinfrastructure.org/projects/7530"><img alt="OpenSSF Best Practices" src="https://bestpractices.coreinfrastructure.org/projects/7530/badge" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/sphinx.yml"><img alt="Sphinx build" src="https://github.com/NL-BioImaging/biomero/actions/workflows/sphinx.yml/badge.svg?branch=main" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/pages/pages-build-deployment"><img alt="pages-build-deployment" src="https://github.com/NL-BioImaging/biomero/actions/workflows/pages/pages-build-deployment/badge.svg" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/python-package.yml"><img alt="python-package build" src="https://github.com/NL-BioImaging/biomero/actions/workflows/python-package.yml/badge.svg" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/python-publish.yml"><img alt="python-publish build" src="https://github.com/NL-BioImaging/biomero/actions/workflows/python-publish.yml/badge.svg?branch=main" /></a></p>
+<p><a class="reference external" href="https://opensource.org/licenses/Apache-2.0"><img alt="License" src="https://img.shields.io/badge/License-Apache_2.0-blue.svg" /></a> <a class="reference external" href="https://zenodo.org/badge/latestdoi/638954891"><img alt="DOI" src="https://zenodo.org/badge/638954891.svg" /></a> <a class="reference external" href="https://pypi.org/project/biomero/"><img alt="PyPI - Version" src="https://img.shields.io/pypi/v/biomero" /></a> <a class="reference external" href="https://pypi.org/project/biomero/"><img alt="PyPI - Python Versions" src="https://img.shields.io/pypi/pyversions/biomero" /></a> <img alt="Slurm" src="https://img.shields.io/badge/Slurm-21.08.6-blue.svg" /> <img alt="OMERO" src="https://img.shields.io/badge/OMERO-5.6.8-blue.svg" /> <a class="reference external" href="https://fair-software.eu"><img alt="fair-software.eu" src="https://img.shields.io/badge/fair--software.eu-%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F%20%20%E2%97%8F-green" /></a> <a class="reference external" href="https://bestpractices.coreinfrastructure.org/projects/7530"><img alt="OpenSSF Best Practices" src="https://bestpractices.coreinfrastructure.org/projects/7530/badge" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/sphinx.yml"><img alt="Sphinx build" src="https://github.com/NL-BioImaging/biomero/actions/workflows/sphinx.yml/badge.svg?branch=main" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/pages/pages-build-deployment"><img alt="pages-build-deployment" src="https://github.com/NL-BioImaging/biomero/actions/workflows/pages/pages-build-deployment/badge.svg" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/python-package.yml"><img alt="python-package build" src="https://github.com/NL-BioImaging/biomero/actions/workflows/python-package.yml/badge.svg" /></a> <a class="reference external" href="https://github.com/NL-BioImaging/biomero/actions/workflows/python-publish.yml"><img alt="python-publish build" src="https://github.com/NL-BioImaging/biomero/actions/workflows/python-publish.yml/badge.svg?branch=main" /></a> <a class="reference external" href="https://coveralls.io/github/NL-BioImaging/biomero?branch=main"><img alt="Coverage Status" src="https://coveralls.io/repos/github/NL-BioImaging/biomero/badge.svg?branch=main" /></a></p>
 <p>The <strong>BIOMERO</strong> framework, for <strong>B</strong>io<strong>I</strong>mage analysis in <strong>OMERO</strong>, allows you to run (FAIR) bioimage analysis workflows directly from OMERO on a high-performance compute (HPC) cluster, remotely through SSH.</p>
 <p>The BIOMERO framework consists of this Python library <code class="docutils literal notranslate"><span class="pre">biomero</span></code>, together with the <a class="reference external" href="https://github.com/NL-BioImaging/biomero-scripts">BIOMERO scripts</a> that can be run directly from the OMERO web interface.</p>
 <p>The package includes the <code class="docutils literal notranslate"><span class="pre">SlurmClient</span></code> class, which provides <strong>SSH-based connectivity</strong> and interaction with a <a class="reference external" href="https://slurm.schedmd.com/quickstart.html">Slurm</a> (high-performance compute) cluster. The package enables users to submit jobs, monitor job status, retrieve job output, and perform other Slurm-related tasks. Additionally, the package offers functionality for configuring and managing paths to Slurm data and Singularity images (think Docker containers…), as well as specific FAIR image analysis workflows and their associated repositories.</p>
@@ -200,7 +200,7 @@ <h2>OMERO Requirements<a class="headerlink" href="#omero-requirements" title="Pe
 <ol class="arabic simple">
 <li><p>SSH client and access to the Slurm cluster (w/ private key / headless)</p></li>
 <li><p>SCP access to the Slurm cluster</p></li>
-<li><p>Python3.7+</p></li>
+<li><p>Python3.8+</p></li>
 <li><p>This library installed</p>
 <ul class="simple">
 <li><p>Latest release on PyPI <code class="docutils literal notranslate"><span class="pre">python3</span> <span class="pre">-m</span> <span class="pre">pip</span> <span class="pre">install</span> <span class="pre">biomero</span></code></p></li>
@@ -546,7 +546,7 @@ <h1>How to add your new custom workflow<a class="headerlink" href="#how-to-add-y
 <ol class="arabic simple">
 <li><p>Rewrite your script to be headless / to be executable on the commandline. This requires handling of commandline parameters as input.</p>
 <ul class="simple">
-<li><p>Make sure the I/O matches the Slurm job, see <a class="reference external" href="https://docs.python.org/3/library/io.html#module-io" title="Python 3.12"><span class="xref myst">previous chapter</span></a>.</p></li>
+<li><p>Make sure the I/O matches the Slurm job, see <a class="reference external" href="https://docs.python.org/3/library/io.html#module-io" title="Python 3.13"><span class="xref myst">previous chapter</span></a>.</p></li>
 </ul>
 </li>
 <li><p>Describe these commandline parameters in a <code class="docutils literal notranslate"><span class="pre">descriptor.json</span></code> (see previous <a class="reference internal" href="#workflow-metadata-via-descriptorjson"><span class="xref myst">chapter</span></a>). E.g. <a class="reference external" href="https://doc.uliege.cytomine.org/dev-guide/algorithms/write-app#create-the-json-descriptor">like this</a>.</p></li>

tutorial_link.html

@@ -1990,10 +1990,10 @@ <h2>5. Setting up (BI)OMERO in Azure too (Optional)<a class="headerlink" href="#
 <li><p>Ready! <code class="docutils literal notranslate"><span class="pre">ssh</span> <span class="pre">localslurm</span></code> (or whatever you called the alias)</p></li>
 </ul>
 <ol class="arabic simple" start="4">
-<li><p>Let’s edit the BIOMERO configuration <code class="docutils literal notranslate"><span class="pre">slurm-config.ini</span></code>, located in the worker-processor node</p></li>
+<li><p>Let’s edit the BIOMERO configuration <code class="docutils literal notranslate"><span class="pre">slurm-config.ini</span></code>, located in the biomeroworker node</p></li>
 </ol>
 <ul class="simple">
-<li><p><code class="docutils literal notranslate"><span class="pre">vi</span> <span class="pre">~/NL-BIOMERO/worker-processor/slurm-config.ini</span></code></p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">vi</span> <span class="pre">~/NL-BIOMERO/biomeroworker/slurm-config.ini</span></code></p></li>
 <li><p>Change the <code class="docutils literal notranslate"><span class="pre">host</span></code> if you did not use the <code class="docutils literal notranslate"><span class="pre">localslurm</span></code> alias in the config above.</p></li>
 <li><p>Change ALL the <code class="docutils literal notranslate"><span class="pre">[SLURM]</span></code> paths to match our new slurm setup:</p></li>
 </ul>
@@ -2040,7 +2040,7 @@ <h2>5. Setting up (BI)OMERO in Azure too (Optional)<a class="headerlink" href="#
 <li><p><code class="docutils literal notranslate"><span class="pre">cd</span> <span class="pre">NL-BIOMERO</span></code></p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">compose</span> <span class="pre">down</span></code></p></li>
 <li><p><code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">compose</span> <span class="pre">up</span> <span class="pre">-d</span> <span class="pre">--build</span></code></p></li>
-<li><p>Now <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">logs</span> <span class="pre">-f</span> <span class="pre">nl-biomero-omeroworker-processor-1</span></code> should show some good logs leading to: <code class="docutils literal notranslate"><span class="pre">Starting</span> <span class="pre">node</span> <span class="pre">omeroworker-processor</span></code>.</p></li>
+<li><p>Now <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">logs</span> <span class="pre">-f</span> <span class="pre">nl-biomero-biomeroworker-1</span></code> should show some good logs leading to: <code class="docutils literal notranslate"><span class="pre">Starting</span> <span class="pre">node</span> <span class="pre">biomeroworker</span></code>.</p></li>
 </ul>
 </div>
 <div class="section" id="showtime">
@@ -2184,12 +2184,12 @@ <h2>6. Showtime!<a class="headerlink" href="#showtime" title="Permalink to this
 </div>
 <p>We will do this ad-hoc, by changing the configuration for CellPose in the <code class="docutils literal notranslate"><span class="pre">slurm-config.ini</span></code> in our installation:</p>
 <ul class="simple">
-<li><p>First, edit the config on the main VM with <code class="docutils literal notranslate"><span class="pre">vi</span> <span class="pre">worker-processor/slurm-config.ini</span></code></p></li>
+<li><p>First, edit the config on the main VM with <code class="docutils literal notranslate"><span class="pre">vi</span> <span class="pre">biomeroworker/slurm-config.ini</span></code></p></li>
 <li><p>Add this line to your workflows <code class="docutils literal notranslate"><span class="pre">&lt;wf&gt;_job_cpus-per-task=2</span></code>, e.g. <code class="docutils literal notranslate"><span class="pre">cellpose_job_cpus-per-task=2</span></code></p></li>
 <li><p>save file (<code class="docutils literal notranslate"><span class="pre">:wq</span></code>)</p></li>
 <li><p>Don’t forget to open your .ssh to the container <code class="docutils literal notranslate"><span class="pre">chmod</span> <span class="pre">-R</span> <span class="pre">777</span> <span class="pre">~/.ssh</span></code> (and close it later)</p></li>
-<li><p>Restart the biomero container(s) (<code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">compose</span> <span class="pre">down</span></code> &amp; <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">compose</span> <span class="pre">up</span> <span class="pre">-d</span> <span class="pre">--build</span></code>, perhaps specifically for <code class="docutils literal notranslate"><span class="pre">omeroworker-processor</span></code>).</p></li>
-<li><p>Check logs to see if biomero started up properly <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">logs</span> <span class="pre">-f</span> <span class="pre">nl-biomero-omeroworker-processor-1</span></code></p></li>
+<li><p>Restart the biomero container(s) (<code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">compose</span> <span class="pre">down</span></code> &amp; <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">compose</span> <span class="pre">up</span> <span class="pre">-d</span> <span class="pre">--build</span></code>, perhaps specifically for <code class="docutils literal notranslate"><span class="pre">biomeroworker</span></code>).</p></li>
+<li><p>Check logs to see if biomero started up properly <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">logs</span> <span class="pre">-f</span> <span class="pre">nl-biomero-biomeroworker-1</span></code></p></li>
 </ul>
 <ol class="arabic simple" start="6">
 <li><p>Next, time to segment! Time to spin up those SLURM compute nodes:</p></li>