From 6fe03e749657ab43f60bf9aa45a3d9b1b0df1b5d Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Tue, 17 Dec 2024 22:00:57 +0000
Subject: [PATCH] Deployed 7c802f6 with MkDocs version: 1.6.1

---
 changelog/index.html     | 16 +++++++++++++++-
 search/search_index.json |  2 +-
 2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/changelog/index.html b/changelog/index.html
index 3770785b..8a17eaea 100644
--- a/changelog/index.html
+++ b/changelog/index.html
@@ -72,7 +72,7 @@
     <div data-md-component="skip">
       
         
-        <a href="#740-december-12-2024" class="md-skip">
+        <a href="#741-december-17-2024" class="md-skip">
           Skip to content
         </a>
       
@@ -623,6 +623,15 @@
     </label>
     <ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>
       
+        <li class="md-nav__item">
+  <a href="#741-december-17-2024" class="md-nav__link">
+    <span class="md-ellipsis">
+      7.4.1 (December 17, 2024)
+    </span>
+  </a>
+  
+</li>
+      
         <li class="md-nav__item">
   <a href="#740-december-12-2024" class="md-nav__link">
     <span class="md-ellipsis">
@@ -1087,6 +1096,11 @@
 
   <h1>Changelog</h1>
 
+<h3 id="741-december-17-2024">7.4.1 (December 17, 2024)<a class="headerlink" href="#741-december-17-2024" title="Permanent link">&para;</a></h3>
+<ul>
+<li>Enabled skeleton client skeleton_exists() and generate_bulk_skeletons_async() endpoints to accept a either a list or a numpy array of root ids.</li>
+<li>Eliminated skeleton client generate_bulk_skeletons_async() issue in which passing a large number of root ids would cause a "URI too big" error.</li>
+</ul>
 <h3 id="740-december-12-2024">7.4.0 (December 12, 2024)<a class="headerlink" href="#740-december-12-2024" title="Permanent link">&para;</a></h3>
 <p>annotation module: Added support for "None", "none", etc. to remove notice_text and 
 improved documentation on method</p>
diff --git a/search/search_index.json b/search/search_index.json
index bef9f3aa..93f75de6 100644
--- a/search/search_index.json
+++ b/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"<p>CAVE is short for Connectome Annotation Versioning Engine. CAVE is a set of microservices that provide a framework for storing and versioning connectomics data and large sets of dynamic annotations, metadata, and segmentations. This repository supplies client-side code to easily interact with the microservices in CAVE.</p>"},{"location":"#documentation-layout","title":"Documentation Layout","text":"<p>To learn how to install <code>caveclient</code>, visit Installation.</p> <p>To see hands-on examples of using <code>caveclient</code> in a notebook, visit the Tutorials.</p> <p>To see API reference documentation for interacting with a <code>caveclient.CAVEclient</code> object, visit the Client API.</p> <p>To see a glossary of terms used in the documentation, visit the Glossary (work in progress).</p> <p>For information about how to contribute to the documentation or the package, visit the Contributing page. Feedback on the documentation is welcome! Please open an issue or use the \"Edit this page\" button at the top right of any page to suggest changes.</p> <p>To read a full description of the Connectome Annotation Versioning Engine, including a description of the various services please read this paper.</p>"},{"location":"changelog/","title":"Changelog","text":""},{"location":"changelog/#740-december-12-2024","title":"7.4.0 (December 12, 2024)","text":"<p>annotation module: Added support for \"None\", \"none\", etc. to remove notice_text and  improved documentation on method</p>"},{"location":"changelog/#732-december-10-2024","title":"7.3.2 (December 10, 2024)","text":"<p>Dropped oldest supported numpy requirements and declared python 3.8 support</p>"},{"location":"changelog/#731-december-5-2024","title":"7.3.1 (December 5, 2024)","text":"<p>Improved documentation to annotation.update_table_metadata</p>"},{"location":"changelog/#730-december-5-2024","title":"7.3.0 (December 5, 2024)","text":"<p>Added get_minimal_covering_nodes to chunkedgraph (#274)</p>"},{"location":"changelog/#721-december-3-2024","title":"7.2.1 (December 3, 2024)","text":"<p>Added raw strings to avoid deprecation warning (#275)</p>"},{"location":"changelog/#720-december-3-2024","title":"7.2.0 (December 3, 2024)","text":"<p>Allow get_state_json to accept direct URLs (#271)</p>"},{"location":"changelog/#710-december-3-2024","title":"7.1.0 (December 3, 2024)","text":"<p>Added get_leaves_multiple to chunkedgraph module (#273)</p>"},{"location":"changelog/#700-november-20-2024","title":"7.0.0 (November 20, 2024)","text":"<ul> <li>Simplified skeleton client interface to only accept formats of 'dict' and 'swc'   Structure of the dictionary returned was also changed compared to previous \"json\" format, where the dict is flat with only one level of keys.   Corresponding change made to MeshParty 1.18.0 allows for hydration of MeshParty Skeleton object from this dict format. </li> </ul>"},{"location":"changelog/#650-november-15-2024","title":"6.5.0 (November 15, 2024)","text":"<ul> <li>Added endpoints for bulk skeleton retrieval and querying whether a skeleton exists or not.</li> </ul>"},{"location":"changelog/#641","title":"6.4.1","text":"<ul> <li>Fix networkx deprecation warning.</li> </ul>"},{"location":"changelog/#640-october-31-2024","title":"6.4.0 (October 31, 2024)","text":"<ul> <li>Added a CAVEclientMock function to ease the use of fully mocked CAVEclients in testing.</li> <li>Fix a bug about use of global server.</li> <li>Fix writing of local server secret to be more secure.</li> </ul>"},{"location":"changelog/#630","title":"6.3.0","text":"<ul> <li>Add a convenience function to get a complete L2 feature dataframe.</li> </ul>"},{"location":"changelog/#620","title":"6.2.0","text":"<ul> <li>Fix things so that auth will not error if the secrets directory is read-only.</li> </ul>"},{"location":"changelog/#612","title":"6.1.2","text":"<ul> <li>Fix query filter bug in tables interface.</li> </ul>"},{"location":"changelog/#611","title":"6.1.1","text":"<ul> <li>Fix warnings.</li> </ul>"},{"location":"changelog/#610","title":"6.1.0","text":"<ul> <li>Added support for inequality filters for numerical columns in materialization tables interface.</li> </ul>"},{"location":"changelog/#600-october-14-2024","title":"6.0.0 (October 14, 2024)","text":"<ul> <li>Refactored CAVEclient internals away from a factory syntax. Most users should not notice a difference, but code that relied on specific subclient type logic may be affected.</li> </ul>"},{"location":"changelog/#5250","title":"5.25.0","text":"<ul> <li>Added ability to suppress table warnings in client.materialize.query_table and similar methods by setting <code>log_warnings=False</code></li> </ul>"},{"location":"changelog/#5200-april-8-2024","title":"5.20.0 (April 8, 2024)","text":"<ul> <li>Added generalized support for detecting server versions to provide timely exceptions to users</li> <li>Used new support to check that chunkegraph has updated version before using spatial bounds kwarg   on client.chunkedgraph.level2_chunk_graph</li> <li>Added support for postign and getting segment properties files to client.state</li> </ul>"},{"location":"changelog/#5180","title":"5.18.0","text":"<ul> <li>Added serialization support for pandas.index</li> </ul>"},{"location":"changelog/#5173","title":"5.17.3","text":"<ul> <li>Minor documentation typo fix</li> </ul>"},{"location":"changelog/#5172","title":"5.17.2","text":"<ul> <li>Bug fixes related to table_manager interface</li> </ul>"},{"location":"changelog/#5171","title":"5.17.1","text":"<ul> <li>Bug fixes related to table_manager interface</li> </ul>"},{"location":"changelog/#5170","title":"5.17.0","text":"<ul> <li>Fix attrs in dataframe attributes of client.materialize results to remove numpy arrays to allow concatenation of dataframes</li> <li>Added getting multiple schemas in one call to improve initialization of table_manager interface of materialization</li> </ul>"},{"location":"changelog/#5161","title":"5.16.1","text":"<ul> <li>Bugfix on client.chunkedgrpah.level2_chunk_graph</li> </ul>"},{"location":"changelog/#5160","title":"5.16.0","text":"<ul> <li>Added bounding box query to client.chunkedgraph.level2_chunk_graph</li> <li>Fix default materialization version client when server not advertising correctly</li> </ul>"},{"location":"changelog/#5151-jan-18-2024","title":"5.15.1 (Jan 18, 2024)","text":"<ul> <li>minor improvements to release process</li> </ul>"},{"location":"changelog/#5150-jan-18-2024","title":"5.15.0 (Jan 18, 2024)","text":"<ul> <li>Improved documentation with types</li> <li>Improved testing on more python versions</li> <li>Bugfixes for pyton 3.12 compatability</li> </ul>"},{"location":"changelog/#5140-november-24-2023","title":"5.14.0 (November 24, 2023)","text":"<ul> <li>Made automatic detection of neuroglancer versioning when constructing link shortener links</li> </ul>"},{"location":"changelog/#5130-october-26-2023","title":"5.13.0 (October 26, 2023)","text":"<ul> <li>Add option to get expired versions to client.materialize.get_versions</li> </ul>"},{"location":"changelog/#5121-october-16-2023","title":"5.12.1 (October 16, 2023)","text":"<ul> <li>Bugfixes for client.chunkedgraph.get_latest_roots</li> </ul>"},{"location":"changelog/#5120-october-16-2023","title":"5.12.0 (October 16, 2023)","text":"<ul> <li>Improved logic for client.chunkedgraph.get_latest_roots to work forward or backwards in time</li> </ul>"},{"location":"changelog/#5110-september-19-2023","title":"5.11.0 (September 19, 2023)","text":"<ul> <li>Added filter_regex_dict options to client.materialize.query_table interface</li> </ul>"},{"location":"changelog/#5102-august-162023","title":"5.10.2 (August 16,2023)","text":"<ul> <li>Fixed pyarrow support for live_live query</li> </ul>"},{"location":"changelog/#5101-august-142023","title":"5.10.1 (August 14,2023)","text":"<ul> <li>Changed random_sample argument to be an integer number of annotations rather than a floating fraction of table</li> <li>Added option to live_query</li> </ul>"},{"location":"changelog/#590-august-14-2023","title":"5.9.0 (August 14, 2023)","text":"<ul> <li>Added support for native pyarrow deserialization, allowing upgrade to pyarrow version</li> </ul>"},{"location":"changelog/#580","title":"5.8.0","text":"<ul> <li>Allowed int64 root ids to serialize properly</li> <li>Added warning that client.materialize.tables interface is in beta</li> </ul>"},{"location":"changelog/#570","title":"5.7.0","text":"<ul> <li>Fix to ensure stop_layer is at least 1</li> <li>Added client.chunkedgraph.suggest_latest_roots</li> </ul>"},{"location":"changelog/#560","title":"5.6.0","text":"<ul> <li>Added views to client.materialize.tables interface</li> <li>Added optional argument to allow invalid root ids when querying live live, versus creating an exception</li> </ul>"},{"location":"changelog/#551","title":"5.5.1","text":"<ul> <li>documentation fixes on client.materialize.join_query</li> </ul>"},{"location":"changelog/#550","title":"5.5.0","text":"<ul> <li>added methods for different neuroglancer state formats to client.state.</li> </ul>"},{"location":"changelog/#543","title":"5.4.3","text":"<ul> <li>Added 'view' querying options to materialization</li> <li>Added client.materialize.tables interface</li> <li>Added client.materialize.get_tables_metadata to get all metadata in one call</li> </ul>"},{"location":"changelog/#520","title":"5.2.0","text":"<ul> <li>Added local caching of datastack names &gt; server_address to simplify initialization of clients   with servers other than global.daf-apis.com.</li> </ul> <p>Cache is saved on a local file ~/.cloudvolume/secrets/cave_datastack_to_server_map.json</p> <p>Cache will populate the first time caveclient.CAVEclient('my_datastack', server_address=\"https://my_server.com\") is called. Subsequent calls can then just be caveclient.CAVEclient('my_datastack').</p>"},{"location":"changelog/#510","title":"5.1.0","text":"<ul> <li>Added get_oldest_timestamp call to chunkedgraph</li> </ul>"},{"location":"changelog/#501","title":"5.0.1","text":"<ul> <li>Fixed bug with desired_resolution being set at the client level   was being ignored in &gt;5.0.0</li> </ul>"},{"location":"changelog/#500","title":"5.0.0","text":"<ul> <li>Added support for the new CAVE Materialization 3.0 API   Includes support for the new materialization API, which allows for   server side conversion of the units of position, and ensures that   all positions are returned with the same units, even after joins.</li> <li>Added support for querying databases that were materialized without merging   tables together. This will allow for faster materializations.</li> <li>Removed support for LiveLive query from the Materialization 2.0 API client.   Note.. &lt;5.0.0 clients interacting with MaterializationEngine &gt;4.7.0 servers will   use live live query but will doubly convert the units of position if you ask   for a desired resolution, as the old client will also do a conversion server side.</li> <li>Fixed interaction with api version querying of servers from individual   clients to work with verify=False. (useful for testing)</li> <li>Stored infromation from client about mapping between dataframe and table names   and original column names.</li> <li>Added support for suffixes and select columns to be passed by dictionary rather than list   making the selection an application of suffixes more explicit when there are collisions   between column names in joined tables.</li> </ul>"},{"location":"changelog/#older-upgrade-notes","title":"Older Upgrade Notes","text":"<p>Change all select_column calls to pass dictionaries rather than lists. Change all suffix calls to pass dictionaries rather than lists. Advocate for your server administrator to upgrade to MaterializationEngine 4.7.0 or later, so you can use the new MaterializationEngine 3.0 API and client.</p>"},{"location":"contributing/","title":"Contributing","text":"<p>Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.</p> <p>You can contribute in many ways:</p>"},{"location":"contributing/#types-of-contributions","title":"Types of Contributions","text":""},{"location":"contributing/#report-bugs","title":"Report Bugs","text":"<p>Report bugs to our issues page.</p> <p>If you are reporting a bug, please include:</p> <ul> <li>Your operating system name and version.</li> <li>Any details about your local setup that might be helpful in troubleshooting.</li> <li>Detailed steps to reproduce the bug, in the form of a minimal reproducible example.</li> </ul>"},{"location":"contributing/#fix-bugs","title":"Fix Bugs","text":"<p>Look through the GitHub issues for bugs. Anything tagged with \"bug\" and \"help wanted\" is open to whoever wants to implement it.</p>"},{"location":"contributing/#implement-features","title":"Implement Features","text":"<p>Look through the GitHub issues for features. Anything tagged with \"enhancement\" and \"help wanted\" is open to whoever wants to implement it.</p>"},{"location":"contributing/#write-documentation","title":"Write Documentation","text":"<p><code>caveclient</code> could always use more documentation, whether as part of the official <code>caveclient</code> docs, in docstrings, or even on the web in blog posts, articles, and such.</p>"},{"location":"contributing/#submit-feedback","title":"Submit Feedback","text":"<p>The best way to send feedback is to create an issue on GitHub.</p> <p>If you are proposing a feature:</p> <ul> <li>Explain in detail how it would work.</li> <li>Keep the scope as narrow as possible, to make it easier to implement.</li> <li>Remember that while contributions are welcome, developer/maintainer time is limited.</li> </ul>"},{"location":"contributing/#modifying-code","title":"Modifying Code","text":"<p>Ready to contribute? Here's how to set up <code>caveclient</code> for local development.</p> <ul> <li>Fork the repo on GitHub.</li> <li>Clone your fork locally</li> </ul> <pre><code>git clone git@github.com:your_name_here/CAVEclient.git\n</code></pre> <ul> <li>We use <code>uv</code> for various developer tasks. Ensure you have <code>uv</code> installed according to the installation instructions.</li> </ul> <p>Note</p> <p>While we recommend using <code>uv</code> as described here, these tasks could also be achieved using <code>pip</code> to install and run the various required tools. You can view the development requirements and build/check commands in our <code>pyproject.toml</code>, so we avoid duplicating them here.</p> <ul> <li>Navigate to the newly cloned directory, e.g.:</li> </ul> <pre><code>cd CAVEclient\n</code></pre> <ul> <li>Create a synced virtual environment, optionally specifying a Python version:</li> </ul> <pre><code>uv sync --python 3.12\n</code></pre> <ul> <li>Create a branch for local development:</li> </ul> <pre><code>git checkout -b name-of-your-bugfix-or-feature\n</code></pre> <ul> <li> <p>Make your changes locally</p> </li> <li> <p>If you have added code that should be tested, add tests.</p> </li> <li> <p>If you have modified dependencies in any way, make sure to run</p> </li> </ul> <pre><code>uv sync\n</code></pre> <ul> <li>Make sure you have added documentation for any additions or modifications to public functions or classes. You can build the documentation locally to make sure it renders correctly with:</li> </ul> <pre><code>uvx --from poethepoet poe doc-build\n</code></pre>"},{"location":"contributing/#automated-checks","title":"Automated Checks","text":"<ul> <li>Run the autoformatter:</li> </ul> <pre><code>uvx --from poethepoet poe lint-fix\n</code></pre> <ul> <li>Ensure that your changes pass the checks that will be run on Github Actions, including building the documentation, checking the formatting of the code, and running the tests. To run all at once, do:</li> </ul> <pre><code>uvx --from poethepoet poe checks\n</code></pre> <ul> <li> <p>You may be interested in running some of these checks individually, such as:</p> <ul> <li>To run the tests:</li> </ul> <pre><code>uvx --from poethepoet poe test\n</code></pre> <ul> <li>To build the documentation:</li> </ul> <pre><code>uvx --from poethepoet poe doc-build\n</code></pre> <ul> <li>To run the linter</li> </ul> <pre><code>uvx --from poethepoet poe lint\n</code></pre> </li> </ul>"},{"location":"contributing/#submitting-a-pull-request","title":"Submitting a Pull Request","text":"<ul> <li>Ensure your code has passed all of the tests described above.</li> <li>Commit your changes and push your branch to GitHub:</li> </ul> <pre><code>git add .\ngit commit -m \"Your detailed description of your changes.\"\ngit push origin name-of-your-bugfix-or-feature\n</code></pre> <ul> <li>Submit a pull request through the GitHub website.</li> </ul> <p>Before you submit a pull request, check that it meets these guidelines:</p> <ul> <li>The pull request should include tests if adding a new feature.</li> <li> <p>The docs should be updated with whatever changes you have made. Put   your new functionality into a function with a docstring, and make sure the new   functionality is documented after building the documentation (described above).</p> </li> <li> <p>Once you submit a pull request, automated checks will run. You may require administrator approval before running these checks if this is your first time contributing to the repo.</p> </li> </ul>"},{"location":"glossary/","title":"Glossary","text":"<p>Warning</p> <p>This glossary is a work in progress; for now we are documenting the commonly used  terms that we need to define. Please feel free to contribute definitions or  additional terms.</p> <ul> <li>Datastack</li> <li>Voxel resolution</li> <li>Segmentation resolution</li> <li>MIP</li> <li>Segmentation</li> <li>View</li> </ul>"},{"location":"installation/","title":"Installation","text":""},{"location":"installation/#stable-release","title":"Stable release","text":"<p>To install <code>caveclient</code>, run this command in your terminal:</p> <pre><code>pip install caveclient\n</code></pre> <p>This is the preferred method to install <code>caveclient</code>, as it will always install the most recent stable release.</p> <p>You can also specify a particular version, e.g.</p> <pre><code>pip install caveclient==5.0.0\n</code></pre> <p>If you don't have pip installed, this Python installation guide can guide you through the process.</p>"},{"location":"installation/#from-source","title":"From source","text":"<p>The source for <code>caveclient</code> can be downloaded from the Github repo.</p> <p>You can either clone the public repository:</p> <pre><code>git clone git://github.com/CAVEconnectome/CAVEclient\n</code></pre> <p>Or download the tarball:</p> <pre><code>curl -OJL https://github.com/CAVEconnectome/CAVEclient/tarball/master\n</code></pre> <p>Once you have a copy of the source, you can install it with:</p> <pre><code>pip install .\n</code></pre> <p>Or in editable mode, it can be installed with:</p> <pre><code>pip install -e .\n</code></pre>"},{"location":"api/","title":"Overview","text":"<p>The most common method of interacting with the CAVE Framework is by instantiating a client (<code>caveclient.CAVEclient</code>) and then using that client to interact with various services. Under the hood, the CAVEclient is a collection of individual clients, which can be accessed via properties. For example, to access the materialization client, you can use <code>client.materialize</code>, which (up to the exact version) will actually return a <code>MaterializationClient</code> object.</p> <p>These pages describe the functionality of each of the individual clients, but note that some features may not be available if the server you are accessing is using older versions of particular services.</p>"},{"location":"api/annotation/","title":"client.annotation","text":"<p>Client for interacting with the annotation engine.</p> <p>Methods:</p> Name Description <code>create_table</code> <p>Creates a new data table based on an existing schema</p> <code>delete_annotation</code> <p>Delete one or more annotations in a table. Annotations that are</p> <code>delete_table</code> <p>Marks a table for deletion requires super admin privileges</p> <code>get_annotation</code> <p>Retrieve an annotation or annotations by id(s) and table name.</p> <code>get_annotation_count</code> <p>Get number of annotations in a table</p> <code>get_table_metadata</code> <p>Get metadata about a table</p> <code>get_tables</code> <p>Gets a list of table names for a aligned_volume_name</p> <code>post_annotation</code> <p>Post one or more new annotations to a table in the AnnotationEngine.</p> <code>post_annotation_df</code> <p>Post one or more new annotations to a table in the AnnotationEngine.</p> <code>process_position_columns</code> <p>Process a dataframe into a list of dictionaries</p> <code>raise_for_status</code> <p>Raises requests.HTTPError, if one occurred.</p> <code>stage_annotations</code> <p>Get a StagedAnnotations object to help produce correctly formatted annotations for a given table or schema.</p> <code>update_annotation</code> <p>Update one or more new annotations to a table in the AnnotationEngine.</p> <code>update_annotation_df</code> <p>Update one or more annotations to a table in the AnnotationEngine using a</p> <code>update_metadata</code> <p>Update the metadata on an existing table</p> <code>upload_staged_annotations</code> <p>Upload annotations directly from an Annotation Guide object.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.create_table","title":"<code>create_table(table_name, schema_name, description, voxel_resolution, reference_table=None, track_target_id_updates=None, flat_segmentation_source=None, user_id=None, aligned_volume_name=None, write_permission='PRIVATE', read_permission='PUBLIC', notice_text=None)</code>","text":"<p>Creates a new data table based on an existing schema</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the new table. Cannot be the same as an existing table</p> required <code>schema_name</code> <code>str</code> <p>Name of the schema for the new table.</p> required <code>description</code> <code>str</code> <p>Human readable description for what is in the table. Should include information about who generated the table What data it covers, and how it should be interpreted. And who should you talk to if you want to use it. An Example: a manual synapse table to detect chandelier synapses on 81 PyC cells with complete AISs [created by Agnes - agnesb@alleninstitute.org, uploaded by Forrest]</p> required <code>voxel_resolution</code> <code>List[float]</code> <p>voxel resolution points will be uploaded in, typically nm, i.e [1,1,1] means nanometers [4,4,40] would be 4nm, 4nm, 40nm voxels</p> required <code>reference_table</code> <code>str</code> <p>If the schema you are using is a reference schema Meaning it is an annotation of another annotation. Then you need to specify what the target table those annotations are in.</p> <code>None</code> <code>track_target_id_updates</code> <code>bool</code> <p>Indicates whether to automatically update reference table's foreign key if target annotation table row is updated.</p> <code>None</code> <code>flat_segmentation_source</code> <code>str</code> <p>the source to a flat segmentation that corresponds to this table i.e. precomputed:\\gs:\\mybucket      his_tables_annotation</p> <code>None</code> <code>user_id</code> <code>int</code> <p>If you are uploading this schema on someone else's behalf and you want to link this table with their ID, you can specify it here Otherwise, the table will be created with your userID in the user_id column.</p> <code>None</code> <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <code>write_permission</code> <code>str</code> <p>What permissions to give the table for writing.  One of PRIVATE: only you can write to this table (DEFAULT) GROUP: only members that share a group with you can write (excluding some groups) PUBLIC: Anyone can write to this table. Note all data is logged, and deletes are done by marking rows as deleted, so all data is always recoverable</p> <code>'PRIVATE'</code> <code>read_permission</code> <code>str</code> <p>What permissions to give the table for reading. One of PRIVATE: only you can read this table. Intended to be used for sorting out bugs. GROUP: only members that share a group with you can read (intended for within group vetting) PUBLIC: anyone with permissions to read this datastack can read this data (DEFAULT)</p> <code>'PUBLIC'</code> <code>notice_text</code> <code>str</code> <p>Text the user will see when querying this table. Can be used to warn users of flaws, and uncertainty in the data, or to advertise citations that should be used with this table. Defaults to None, no text. If you want to remove text, send empty string.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p> <p>Examples:</p> <p>Basic annotation table:</p> <pre><code>description = \"Some description about the table\"\nvoxel_res = [4,4,40]\nclient.create_table(\"some_synapse_table\", \"synapse\", description, voxel_res)\n</code></pre>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.delete_annotation","title":"<code>delete_annotation(table_name, annotation_ids, aligned_volume_name=None)</code>","text":"<p>Delete one or more annotations in a table. Annotations that are deleted are recorded as 'non-valid' but are not physically removed from the table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>annotation_ids</code> <code>(dict or list)</code> <p>A list of (or a single) dict of schematized annotation data matching the target table. each dict must contain an \"id\" field which is the ID of the annotation to update</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON: a list of new annotation IDs.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.delete_table","title":"<code>delete_table(table_name, aligned_volume_name=None)</code>","text":"<p>Marks a table for deletion requires super admin privileges</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_annotation","title":"<code>get_annotation(table_name, annotation_ids, aligned_volume_name=None)</code>","text":"<p>Retrieve an annotation or annotations by id(s) and table name.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table</p> required <code>annotation_ids</code> <code>int or iterable</code> <p>ID or IDS of the annotation to retreive</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>Annotation data</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_annotation_count","title":"<code>get_annotation_count(table_name, aligned_volume_name=None)</code>","text":"<p>Get number of annotations in a table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>int</code> <p>number of annotations</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_table_metadata","title":"<code>get_table_metadata(table_name, aligned_volume_name=None)</code>","text":"<p>Get metadata about a table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>metadata about table</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_tables","title":"<code>get_tables(aligned_volume_name=None)</code>","text":"<p>Gets a list of table names for a aligned_volume_name</p> <p>Parameters:</p> Name Type Description Default <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>List of table names</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.post_annotation","title":"<code>post_annotation(table_name, data, aligned_volume_name=None)</code>","text":"<p>Post one or more new annotations to a table in the AnnotationEngine. All inserted annotations will be marked as 'valid'. To invalidate annotations refer to 'update_annotation', 'update_annotation_df' and 'delete_annotation' methods.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>data</code> <code>(dict or list)</code> <p>A list of (or a single) dict of schematized annotation data matching the target table.</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.post_annotation_df","title":"<code>post_annotation_df(table_name, df, position_columns, aligned_volume_name=None)</code>","text":"<p>Post one or more new annotations to a table in the AnnotationEngine. All inserted annotations will be marked as 'valid'. To invalidate annotations see 'update_annotation', 'update_annotation_df' and 'delete_annotation' methods.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>df</code> <code>DataFrame</code> <p>A pandas dataframe containing the annotations. Columns should be fields in schema, position columns need to be called out in position_columns argument.</p> required <code>position_columns</code> <code>Optional[Union[Iterable[str], Mapping[str, str]]]</code> <p>if None, will look for all columns with 'X_position' in the name and assume they go in fields called \"X\". if Iterable assumes each column given ends in _position. (i.e. ['pt_position'] if 'pt' is the name of the position field in schema) if Mapping, keys are names of columns in dataframe, values are the names of the fields (i.e. {'pt_column': 'pt'} would be correct if you had one column named 'pt_column' which needed to go into a schema with a position column called 'pt')</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.process_position_columns","title":"<code>process_position_columns(df, position_columns)</code>  <code>staticmethod</code>","text":"<p>Process a dataframe into a list of dictionaries</p> <p>Parameters:</p> Name Type Description Default <code>df</code> <code>DataFrame</code> <p>Dataframe to process</p> required <code>position_columns</code> <code>Optional[Union[Iterable[str], Mapping[str, str]]]</code> <p>See <code>.post_annotation_df</code></p> required <p>Returns:</p> Type Description <code>dict</code> <p>Annotations ready for posting</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.raise_for_status","title":"<code>raise_for_status(r, log_warning=True)</code>  <code>staticmethod</code>","text":"<p>Raises requests.HTTPError, if one occurred.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.stage_annotations","title":"<code>stage_annotations(table_name=None, schema_name=None, update=False, id_field=False, table_resolution=None, annotation_resolution=None)</code>","text":"<p>Get a StagedAnnotations object to help produce correctly formatted annotations for a given table or schema. StagedAnnotation objects can be uploaded directly with <code>upload_staged_annotations</code>.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Table name to stage annotations for, by default None.</p> <code>None</code> <code>schema_name</code> <code>str</code> <p>Schema name to use to make annotations. Only needed if the table_name is not set, by default None</p> <code>None</code> <code>update</code> <code>bool</code> <p>Set to True if individual annotations are going to be updated, by default False.</p> <code>False</code> <code>id_field</code> <code>bool</code> <p>Set to True if id fields are to be specified. Not needed if update is True, which always needs id fields. Optional, by default False</p> <code>False</code> <code>table_resolution</code> <code>list - like or None</code> <p>Voxel resolution of spatial points in the table in nanometers. This is found automatically from the info service if a table name is provided, by default None. If annotation_resolution is also set, this allows points to be scaled correctly for the table.</p> <code>None</code> <code>annotation_resolution</code> <code>list - like</code> <p>Voxel resolution of spatial points provided by the user when creating annotations. If the table resolution is also available (manually or from the info service), annotations are correctly rescaled for the volume. By default, None.</p> <code>None</code>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.update_annotation","title":"<code>update_annotation(table_name, data, aligned_volume_name=None)</code>","text":"<p>Update one or more new annotations to a table in the AnnotationEngine. Updating is implemented by invalidating the old annotation and inserting a new annotation row, which will receive a new primary key ID.</p> Notes <p>If annotations ids were user provided upon insertion the database will autoincrement from the current max id in the table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>data</code> <code>(dict or list)</code> <p>A list of (or a single) dict of schematized annotation data matching the target table. each dict must contain an \"id\" field which is the ID of the annotation to update</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON: a list of new annotation IDs.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.update_annotation_df","title":"<code>update_annotation_df(table_name, df, position_columns, aligned_volume_name=None)</code>","text":"<p>Update one or more annotations to a table in the AnnotationEngine using a dataframe as format. Updating is implemented by invalidating the old annotation and inserting a new annotation row, which will receive a new primary key ID.</p> Notes <p>If annotations ids were user provided upon insertion the database will autoincrement from the current max id in the table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>df</code> <code>DataFrame</code> <p>A pandas dataframe containing the annotations. Columns should be fields in schema, position columns need to be called out in position_columns argument.</p> required <code>position_columns</code> <code>Iterable[str] or Mapping[str, str] or None</code> <p>if None, will look for all columns with 'X_position' in the name and assume they go in fields called \"X\". if Iterable assumes each column given ends in _position. (i.e. ['pt_position'] if 'pt' is the name of the position field in schema) if Mapping, keys are names of columns in dataframe, values are the names of the fields (i.e. {'pt_column': 'pt'} would be correct if you had one column named 'pt_column' which needed to go into a schema with a position column called 'pt')</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.update_metadata","title":"<code>update_metadata(table_name, description=None, flat_segmentation_source=None, read_permission=None, write_permission=None, user_id=None, notice_text=None, aligned_volume_name=None)</code>","text":"<p>Update the metadata on an existing table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> required <code>description</code> <code>str</code> <p>Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>flat_segmentation_source</code> <code>str</code> <p>Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>read_permission</code> <code>str</code> <p>What permissions to give the table for reading. One of PRIVATE: only you can read this table. Intended to be used for sorting out bugs. GROUP: only members that share a group with you can read (intended for within group vetting) PUBLIC: anyone with permissions to read this datastack can read this data Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>write_permission</code> <code>str</code> <p>What permissions to give the table for writing.  One of PRIVATE: only you can write to this table GROUP: only members that share a group with you can write (excluding some groups) PUBLIC: Anyone can write to this table. Note all data is logged, and deletes are done by marking rows as deleted, so all data is always recoverable Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>user_id</code> <code>int</code> <p>Note, if you use this you will not be able to update the metadata on this table any longer and depending on permissions may not be able to read or write to it Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>notice_text</code> <code>str</code> <p>Text the user will see when querying this table. Can be used to warn users of flaws, and uncertainty in the data, or to advertise citations that should be used with this table. If you wish to remove the notice_text pass an empty string, or \"none\", \"None\", \"NONE\", or any other capitaliztion of the word \"none\". Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.upload_staged_annotations","title":"<code>upload_staged_annotations(staged_annos, aligned_volume_name=None)</code>","text":"<p>Upload annotations directly from an Annotation Guide object. This method uses the options specified in the object, including table name and if the annotation is an update or not.</p> <p>Parameters:</p> Name Type Description Default <code>staged_annos</code> <code>AnnotationGuide</code> <p>AnnotationGuide object with a specified table name and a collection of annotations already filled in.</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>List or dict</code> <p>If new annotations are posted, a list of ids. If annotations are being updated, a dictionary with the mapping from old ids to new ids.</p>"},{"location":"api/auth/","title":"client.auth","text":"<p>Client to find and use auth tokens to access the  other services.</p> <p>Methods:</p> Name Description <code>get_group_users</code> <p>Get users in a group</p> <code>get_new_token</code> <p>Currently, returns instructions for getting a new token based on the current settings and saving it to the local environment. New OAuth tokens are currently not able to be retrieved programmatically.</p> <code>get_token</code> <p>Load a token with a given key the specified token file</p> <code>get_tokens</code> <p>Get the tokens setup for this users</p> <code>get_user_information</code> <p>Get user data.</p> <code>save_token</code> <p>Conveniently save a token in the correct format.</p> <code>setup_token</code> <p>Currently, returns instructions for getting your auth token based on the current settings and saving it to the local environment.</p> <p>Attributes:</p> Name Type Description <code>request_header</code> <p>Formatted request header with the specified token</p> <code>token</code> <p>Secret token used to authenticate yourself to the Connectome Annotation Versioning Engine services.</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.request_header","title":"<code>request_header</code>  <code>property</code>","text":"<p>Formatted request header with the specified token</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.token","title":"<code>token</code>  <code>property</code> <code>writable</code>","text":"<p>Secret token used to authenticate yourself to the Connectome Annotation Versioning Engine services.</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_group_users","title":"<code>get_group_users(group_id)</code>","text":"<p>Get users in a group</p> <p>Parameters:</p> Name Type Description Default <code>group_id</code> <code>int</code> <p>ID value for a given group</p> required <p>Returns:</p> Type Description <code>list</code> <p>List of dicts of user ids. Returns empty list if group does not exist.</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_new_token","title":"<code>get_new_token(open=False, no_text=False)</code>","text":"<p>Currently, returns instructions for getting a new token based on the current settings and saving it to the local environment. New OAuth tokens are currently not able to be retrieved programmatically.</p> <p>Parameters:</p> Name Type Description Default <code>open</code> <code>bool</code> <p>If True, opens a web browser to the web page where you can generate a new token.</p> <code>False</code>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_token","title":"<code>get_token(token_key=None)</code>","text":"<p>Load a token with a given key the specified token file</p> <p>Parameters:</p> Name Type Description Default <code>token_key</code> <code>str or None</code> <p>key in the token file JSON, by default None. If None, uses 'token'.</p> <code>None</code>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_tokens","title":"<code>get_tokens()</code>","text":"<p>Get the tokens setup for this users</p> <p>Returns:</p> Type Description <code>list[dict]:</code> <p>a list of dictionary of tokens, each with the keys \"id\": the id of this token \"token\": the token (str) \"user_id\": the users id (should be your ID)</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_user_information","title":"<code>get_user_information(user_ids)</code>","text":"<p>Get user data.</p> <p>Parameters:</p> Name Type Description Default <code>user_ids</code> <code>list of int</code> <p>user_ids to look up</p> required"},{"location":"api/auth/#caveclient.auth.AuthClient.save_token","title":"<code>save_token(token=None, token_key=default_token_key, overwrite=False, token_file=None, switch_token=True, write_to_server_file=True, ignore_readonly=True, local_server=True)</code>","text":"<p>Conveniently save a token in the correct format.</p> <p>After getting a new token by following the instructions in <code>authclient.get_new_token()</code>, you can save it with a fully default configuration by running:</p> <p>token = 'my_shiny_new_token'</p> <p>authclient.save_token(token=token)</p> <p>Now on next load, authclient=AuthClient() will make an authclient instance using this token. If you would like to specify more information about the json file where the token will be stored, see the parameters below.</p> <p>Parameters:</p> Name Type Description Default <code>token</code> <code>str</code> <p>New token to save, by default None</p> <code>None</code> <code>token_key</code> <code>str</code> <p>Key for the token in the token_file json, by default \"token\"</p> <code>default_token_key</code> <code>overwrite</code> <code>bool</code> <p>Allow an existing token to be changed, by default False</p> <code>False</code> <code>token_file</code> <code>str</code> <p>Path to the token file, by default None. If None, uses the default file location specified above.</p> <code>None</code> <code>switch_token</code> <code>bool</code> <p>If True, switch the auth client over into using the new token, by default True</p> <code>True</code> <code>write_to_server_file</code> <code>bool</code> <p>If True, will write token to a server specific file to support this machine interacting with multiple auth servers.</p> <code>True</code> <code>ignore_readonly</code> <code>bool</code> <p>If True, will only attempt to save a token if the directory is writeable.</p> <code>True</code> <code>local_server</code> <code>bool</code> <p>If True, saves the token to the local server file as well.</p> <code>True</code>"},{"location":"api/auth/#caveclient.auth.AuthClient.setup_token","title":"<code>setup_token(make_new=True, open=True)</code>","text":"<p>Currently, returns instructions for getting your auth token based on the current settings and saving it to the local environment. New OAuth tokens are currently not able to be retrieved programmatically.</p> <p>Parameters:</p> Name Type Description Default <code>make_new</code> <code>bool</code> <p>If True, will make a new token, else prompt you to open a page to retrieve an existing token.</p> <code>True</code> <code>open</code> <code>bool</code> <p>If True, opens a web browser to the web page where you can retrieve a token.</p> <code>True</code>"},{"location":"api/chunkedgraph/","title":"client.chunkedgraph","text":"<p>Client for interacting with the chunkedgraph.</p> <p>Methods:</p> Name Description <code>get_roots</code> <p>Get the root ID for a list of supervoxels.</p> <code>get_root_id</code> <p>Get the root ID for a specified supervoxel.</p> <code>get_minimal_covering_nodes</code> <p>Get the minimal covering nodes for a list of root IDs.</p> <code>get_merge_log</code> <p>Get the merge log (splits and merges) for an object.</p> <code>get_change_log</code> <p>Get the change log (splits and merges) for an object.</p> <code>get_user_operations</code> <p>Get operation details for a user ID. Currently, this is only available to</p> <code>get_tabular_change_log</code> <p>Get a detailed changelog for neurons.</p> <code>get_leaves_many</code> <p>Get all supervoxels for a list of root IDs.</p> <code>get_leaves</code> <p>Get all supervoxels for a root ID.</p> <code>do_merge</code> <p>Perform a merge on the chunked graph.</p> <code>undo_operation</code> <p>Undo an operation.</p> <code>execute_split</code> <p>Execute a multicut split based on points or supervoxels.</p> <code>preview_split</code> <p>Get supervoxel connected components from a preview multicut split.</p> <code>get_children</code> <p>Get the children of a node in the chunked graph hierarchy.</p> <code>get_contact_sites</code> <p>Get contacts for a root ID.</p> <code>find_path</code> <p>Find a path between two locations on a root ID using the level 2 chunked</p> <code>get_subgraph</code> <p>Get subgraph of root id within a bounding box.</p> <code>level2_chunk_graph</code> <p>Get graph of level 2 chunks, the smallest agglomeration level above supervoxels.</p> <code>remesh_level2_chunks</code> <p>Submit specific level 2 chunks to be remeshed in case of a problem.</p> <code>get_operation_details</code> <p>Get the details of a list of operations.</p> <code>get_lineage_graph</code> <p>Returns the lineage graph for a root ID, optionally cut off in the past or</p> <code>get_latest_roots</code> <p>Returns root IDs that are related to the given <code>root_id</code> at a given</p> <code>get_original_roots</code> <p>Returns root IDs that are the latest successors of a given root ID.</p> <code>is_latest_roots</code> <p>Check whether these root IDs are still a root at this timestamp.</p> <code>suggest_latest_roots</code> <p>Suggest latest roots for a given root id, based on overlap of component</p> <code>is_valid_nodes</code> <p>Check whether nodes are valid for given timestamp range.</p> <code>get_root_timestamps</code> <p>Retrieves timestamps when roots where created.</p> <code>get_past_ids</code> <p>For a set of root IDs, get the list of IDs at a past or future time point</p> <code>get_delta_roots</code> <p>Get the list of roots that have changed between <code>timetamp_past</code> and</p> <code>get_oldest_timestamp</code> <p>Get the oldest timestamp in the database.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p> <code>timestamp</code> <code>Optional[datetime]</code> <p>The default timestamp for queries which expect a timestamp. If None, uses the</p> <code>segmentation_info</code> <p>Complete segmentation metadata</p> <code>base_resolution</code> <p>MIP 0 resolution for voxels assumed by the ChunkedGraph</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.timestamp","title":"<code>timestamp: Optional[datetime.datetime]</code>  <code>property</code> <code>writable</code>","text":"<p>The default timestamp for queries which expect a timestamp. If None, uses the current time.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.segmentation_info","title":"<code>segmentation_info</code>  <code>property</code>","text":"<p>Complete segmentation metadata</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.base_resolution","title":"<code>base_resolution</code>  <code>property</code>","text":"<p>MIP 0 resolution for voxels assumed by the ChunkedGraph</p> <p>Returns:</p> Type Description <code>list</code> <p>3-long list of x/y/z voxel dimensions in nm</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_roots","title":"<code>get_roots(supervoxel_ids, timestamp=None, stop_layer=None)</code>","text":"<p>Get the root ID for a list of supervoxels.</p> <p>Parameters:</p> Name Type Description Default <code>supervoxel_ids</code> <code>list or np.array of int</code> <p>Supervoxel IDs to look up.</p> required <code>timestamp</code> <code>datetime</code> <p>UTC datetime to specify the state of the chunkedgraph at which to query, by default None. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <code>stop_layer</code> <code>int or None</code> <p>If True, looks up IDs only up to a given stop layer. Default is None.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of np.uint64</code> <p>Root IDs containing each supervoxel.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_root_id","title":"<code>get_root_id(supervoxel_id, timestamp=None, level2=False)</code>","text":"<p>Get the root ID for a specified supervoxel.</p> <p>Parameters:</p> Name Type Description Default <code>supervoxel_id</code> <code>int</code> <p>Supervoxel id value</p> required <code>timestamp</code> <code>datetime</code> <p>UTC datetime to specify the state of the chunkedgraph at which to query, by default None. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>int64</code> <p>Root ID containing the supervoxel.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_minimal_covering_nodes","title":"<code>get_minimal_covering_nodes(node_ids)</code>","text":"<p>Get the minimal covering nodes for a list of root IDs.</p> <p>Parameters:</p> Name Type Description Default <code>nodes_ids</code> <code>Iterable of int or np.int64</code> <p>List of root IDs to query.</p> required <p>Returns:</p> Type Description <code>np.array of np.int64:</code> <p>List of PCG node_ids that minimally and exactly cover the input nodes</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_merge_log","title":"<code>get_merge_log(root_id)</code>","text":"<p>Get the merge log (splits and merges) for an object.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID to look up.</p> required <p>Returns:</p> Type Description <code>list</code> <p>List of merge events in the history of the object.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_change_log","title":"<code>get_change_log(root_id, filtered=True)</code>","text":"<p>Get the change log (splits and merges) for an object.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID to look up.</p> required <code>filtered</code> <code>bool</code> <p>Whether to filter the change log to only include splits and merges which affect the final state of the object (<code>filtered=True</code>), as opposed to including edit history for objects which as some point were split from the query object <code>root_id</code> (<code>filtered=False</code>). Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary summarizing split and merge events in the object history, containing the following keys:</p> <ul> <li>\"n_merges\": int<ul> <li>Number of merges</li> </ul> </li> <li>\"n_splits\": int<ul> <li>Number of splits</li> </ul> </li> <li>\"operations_ids\": list of int<ul> <li>Identifiers for each operation</li> </ul> </li> <li>\"past_ids\": list of int<ul> <li>Previous root ids for this object</li> </ul> </li> <li>\"user_info\": dict of dict<ul> <li>Dictionary keyed by user (string) to a dictionary specifying how many   merges and splits that user performed on this object</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_user_operations","title":"<code>get_user_operations(user_id, timestamp_start, include_undo=True, timestamp_end=None)</code>","text":"<p>Get operation details for a user ID. Currently, this is only available to admins.</p> <p>Parameters:</p> Name Type Description Default <code>user_id</code> <code>int</code> <p>User ID to query (use 0 for all users (admin only)).</p> required <code>timestamp_start</code> <code>datetime</code> <p>Timestamp to start filter (UTC).</p> required <code>include_undo</code> <code>bool</code> <p>Whether to include undos. Defaults to True.</p> <code>True</code> <code>timestamp_end</code> <code>datetime</code> <p>Timestamp to end filter (UTC). Defaults to now.</p> <code>None</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>DataFrame including the following columns:</p> <ul> <li>\"operation_id\": int<ul> <li>Identifier for the operation.</li> </ul> </li> <li>\"timestamp\": datetime.datetime<ul> <li>Timestamp of the operation.</li> </ul> </li> <li>\"user_id\": int<ul> <li>User who performed the operation.</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_tabular_change_log","title":"<code>get_tabular_change_log(root_ids, filtered=True)</code>","text":"<p>Get a detailed changelog for neurons.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>list of int</code> <p>Object root IDs to look up.</p> required <code>filtered</code> <code>bool</code> <p>Whether to filter the change log to only include splits and merges which affect the final state of the object (<code>filtered=True</code>), as opposed to including edit history for objects which as some point were split from the query objects in <code>root_ids</code> (<code>filtered=False</code>). Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict of pd.DataFrame</code> <p>The keys are the root IDs, and the values are DataFrames with the following columns and datatypes:</p> <ul> <li>\"operation_id\": int<ul> <li>Identifier for the operation.</li> </ul> </li> <li>\"timestamp\": int<ul> <li>Timestamp of the operation, provided in milliseconds. To convert to datetime, use <code>datetime.datetime.utcfromtimestamp(timestamp/1000)</code>.</li> </ul> </li> <li>\"user_id\": int<ul> <li>User who performed the operation.</li> </ul> </li> <li>\"before_root_ids: list of int<ul> <li>Root IDs of objects that existed before the operation.</li> </ul> </li> <li>\"after_root_ids: list of int<ul> <li>Root IDs of objects created by the operation. Note that this only records the root id that was kept as part of the query object, so there will only be one in this list.</li> </ul> </li> <li>\"is_merge\": bool<ul> <li>Whether the operation was a merge.</li> </ul> </li> <li>\"user_name\": str<ul> <li>Name of the user who performed the operation.</li> </ul> </li> <li>\"user_affiliation\": str<ul> <li>Affiliation of the user who performed the operation.</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_leaves_many","title":"<code>get_leaves_many(root_ids, bounds=None, stop_layer=None)</code>","text":"<p>Get all supervoxels for a list of root IDs.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>Iterable</code> <p>Root IDs to query.</p> required <code>bounds</code> <p>If specified, returns supervoxels within a 3x2 numpy array of bounds <code>[[minx,maxx],[miny,maxy],[minz,maxz]]</code>. If None, finds all supervoxels.</p> <code>None</code> <code>stop_layer</code> <code>int</code> <p>If specified, returns chunkedgraph nodes at layer <code>stop_layer</code> default will be <code>stop_layer=1</code> (supervoxels).</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dict relating ids to contacts</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_leaves","title":"<code>get_leaves(root_id, bounds=None, stop_layer=None)</code>","text":"<p>Get all supervoxels for a root ID.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID to query.</p> required <code>bounds</code> <p>If specified, returns supervoxels within a 3x2 numpy array of bounds <code>[[minx,maxx],[miny,maxy],[minz,maxz]]</code>. If None, finds all supervoxels.</p> <code>None</code> <code>stop_layer</code> <code>int</code> <p>If specified, returns chunkedgraph nodes at layer <code>stop_layer</code> default will be <code>stop_layer=1</code> (supervoxels).</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of np.int64</code> <p>Array of supervoxel IDs (or node ids if <code>stop_layer&gt;1</code>).</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.do_merge","title":"<code>do_merge(supervoxels, coords, resolution=(4, 4, 40))</code>","text":"<p>Perform a merge on the chunked graph.</p> <p>Parameters:</p> Name Type Description Default <code>supervoxels</code> <code>iterable</code> <p>An N-long list of supervoxels to merge.</p> required <code>coords</code> <code>array</code> <p>An Nx3 array of coordinates of the supervoxels in units of <code>resolution</code>.</p> required <code>resolution</code> <code>tuple</code> <p>What to multiply <code>coords</code> by to get nanometers. Defaults to (4,4,40).</p> <code>(4, 4, 40)</code>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.undo_operation","title":"<code>undo_operation(operation_id)</code>","text":"<p>Undo an operation.</p> <p>Parameters:</p> Name Type Description Default <code>operation_id</code> <code>int</code> <p>Operation ID to undo.</p> required <p>Returns:</p> Type Description <code>dict</code>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.execute_split","title":"<code>execute_split(source_points, sink_points, root_id, source_supervoxels=None, sink_supervoxels=None)</code>","text":"<p>Execute a multicut split based on points or supervoxels.</p> <p>Parameters:</p> Name Type Description Default <code>source_points</code> <code>array or list</code> <p>Nx3 list or array of 3d points in nm coordinates for source points (red).</p> required <code>sink_points</code> <code>array or list</code> <p>Mx3 list or array of 3d points in nm coordinates for sink points (blue).</p> required <code>root_id</code> <code>int</code> <p>Root ID of object to do split preview.</p> required <code>source_supervoxels</code> <code>(array, list or None)</code> <p>If providing source supervoxels, an N-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <code>sink_supervoxels</code> <code>(array, list or None)</code> <p>If providing sink supervoxels, an M-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <p>Returns:</p> Name Type Description <code>operation_id</code> <code>int</code> <p>Unique ID of the split operation</p> <code>new_root_ids</code> <code>list of int</code> <p>List of new root IDs resulting from the split operation.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.preview_split","title":"<code>preview_split(source_points, sink_points, root_id, source_supervoxels=None, sink_supervoxels=None, return_additional_ccs=False)</code>","text":"<p>Get supervoxel connected components from a preview multicut split.</p> <p>Parameters:</p> Name Type Description Default <code>source_points</code> <code>array or list</code> <p>Nx3 list or array of 3d points in nm coordinates for source points (red).</p> required <code>sink_points</code> <code>array or list</code> <p>Mx3 list or array of 3d points in nm coordinates for sink points (blue).</p> required <code>root_id</code> <code>int</code> <p>Root ID of object to do split preview.</p> required <code>source_supervoxels</code> <code>(array, list or None)</code> <p>If providing source supervoxels, an N-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <code>sink_supervoxels</code> <code>(array, list or None)</code> <p>If providing sink supervoxels, an M-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <code>return_additional_ccs</code> <code>bool</code> <p>If True, returns any additional connected components beyond the ones with source and sink points. In most situations, this can be ignored. By default, False.</p> <code>False</code> <p>Returns:</p> Name Type Description <code>source_connected_component</code> <code>list</code> <p>Supervoxel IDs in the component with the most source points.</p> <code>sink_connected_component</code> <code>list</code> <p>Supervoxel IDs in the component with the most sink points.</p> <code>successful_split</code> <code>bool</code> <p>True if the split worked.</p> <code>other_connected_components (optional) : list of lists of int</code> <p>List of lists of supervoxel IDs for any other resulting connected components. Only returned if <code>return_additional_ccs</code> is True.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_children","title":"<code>get_children(node_id)</code>","text":"<p>Get the children of a node in the chunked graph hierarchy.</p> <p>Parameters:</p> Name Type Description Default <code>node_id</code> <code>int</code> <p>Node ID to query.</p> required <p>Returns:</p> Type Description <code>np.array of np.int64</code> <p>IDs of child nodes.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_contact_sites","title":"<code>get_contact_sites(root_id, bounds, calc_partners=False)</code>","text":"<p>Get contacts for a root ID.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID to query.</p> required <code>bounds</code> <p>Bounds within a 3x2 numpy array of bounds <code>[[minx,maxx],[miny,maxy],[minz,maxz]]</code> for which to find contacts. Running this query without bounds is too slow.</p> required <code>calc_partners</code> <code>bool</code> <p>If True, get partner root IDs. By default, False.</p> <code>False</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dict relating ids to contacts</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.find_path","title":"<code>find_path(root_id, src_pt, dst_pt, precision_mode=False)</code>","text":"<p>Find a path between two locations on a root ID using the level 2 chunked graph.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID to query.</p> required <code>src_pt</code> <code>array</code> <p>3-element array of xyz coordinates in nm for the source point.</p> required <code>dst_pt</code> <code>array</code> <p>3-element array of xyz coordinates in nm for the destination point.</p> required <code>precision_mode</code> <code>bool</code> <p>Whether to perform the search in precision mode. Defaults to False.</p> <code>False</code> <p>Returns:</p> Name Type Description <code>centroids_list</code> <code>array</code> <p>Array of centroids along the path.</p> <code>l2_path</code> <code>np.array of int</code> <p>Array of level 2 chunk IDs along the path.</p> <code>failed_l2_ids</code> <code>np.array of int</code> <p>Array of level 2 chunk IDs that failed to find a path.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_subgraph","title":"<code>get_subgraph(root_id, bounds)</code>","text":"<p>Get subgraph of root id within a bounding box.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root (or any node ID) of chunked graph to query.</p> required <code>bounds</code> <code>array</code> <p>3x2 bounding box (x,y,z) x (min,max) in chunked graph coordinates.</p> required <p>Returns:</p> Type Description <code>np.array of np.int64</code> <p>Node IDs in the subgraph.</p> <code>np.array of np.double</code> <p>Affinities of edges in the subgraph.</p> <code>np.array of np.int32</code> <p>Areas of nodes in the subgraph.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.level2_chunk_graph","title":"<code>level2_chunk_graph(root_id, bounds=None)</code>","text":"<p>Get graph of level 2 chunks, the smallest agglomeration level above supervoxels.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root id of object</p> required <code>bounds</code> <code>array</code> <p>3x2 bounding box (x,y,z) x (min,max) in chunked graph coordinates (use <code>client.chunkedgraph.base_resolution</code> to view this default resolution for your chunkedgraph client). Note that the result will include any level 2 nodes which have chunk boundaries within some part of this bounding box, meaning that the representative point for a given level 2 node could still be slightly outside of these bounds. If None, returns all level 2 chunks for the root ID.</p> <code>None</code> <p>Returns:</p> Type Description <code>list of list</code> <p>Edge list for level 2 chunked graph. Each element of the list is an edge, and each edge is a list of two node IDs (source and target).</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.remesh_level2_chunks","title":"<code>remesh_level2_chunks(chunk_ids)</code>","text":"<p>Submit specific level 2 chunks to be remeshed in case of a problem.</p> <p>Parameters:</p> Name Type Description Default <code>chunk_ids</code> <code>list</code> <p>List of level 2 chunk IDs.</p> required"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_operation_details","title":"<code>get_operation_details(operation_ids)</code>","text":"<p>Get the details of a list of operations.</p> <p>Parameters:</p> Name Type Description Default <code>operation_ids</code> <code>Iterable[int]</code> <p>List/array of operation IDs.</p> required <p>Returns:</p> Type Description <code>dict of str to dict</code> <p>A dict of dicts of operation info, keys are operation IDs (as strings), values are a dictionary of operation info for the operation. These dictionaries contain the following keys:</p> <ul> <li>\"added_edges\"/\"removed_edges\": list of list of int<ul> <li>List of edges added (if a merge) or removed (if a split) by this operation. Each edge is a list of two supervoxel IDs (source and target).</li> </ul> </li> <li>\"roots\": list of int<ul> <li>List of root IDs that were created by this operation.</li> </ul> </li> <li>\"sink_coords\": list of list of int<ul> <li>List of sink coordinates for this operation. The sink is one of the points placed by the user when specifying the operation. Each sink coordinate is a list of three integers (x, y, z), corresponding to spatial coordinates in segmentation voxel space.</li> </ul> </li> <li>\"source_coords\": list of list of int<ul> <li>List of source coordinates for this operation. The source is one of the points placed by the user when specifying the operation. Each source coordinate is a list of three integers (x, y, z), corresponding to spatial coordinates in segmentation voxel space.</li> </ul> </li> <li>\"timestamp\": str<ul> <li>Timestamp of the operation.</li> </ul> </li> <li>\"user\": str<ul> <li>User ID number who performed the operation (as a string).</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_lineage_graph","title":"<code>get_lineage_graph(root_id, timestamp_past=None, timestamp_future=None, as_nx_graph=False, exclude_links_to_future=False, exclude_links_to_past=False)</code>","text":"<p>Returns the lineage graph for a root ID, optionally cut off in the past or the future.</p> <p>Each change in the chunked graph creates a new root ID for the object after that change. This function returns a graph of all root IDs for a given object, tracing the history of the object in terms of merges and splits.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID.</p> required <code>timestamp_past</code> <code>datetime or None</code> <p>Cutoff for the lineage graph backwards in time. By default, None.</p> <code>None</code> <code>timestamp_future</code> <code>datetime or None</code> <p>Cutoff for the lineage graph going forwards in time. By default, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <code>as_nx_graph</code> <p>If True, a NetworkX graph is returned.</p> <code>False</code> <code>exclude_links_to_future</code> <p>If True, links from nodes before <code>timestamp_future</code> to after <code>timestamp_future</code> are removed. If False, the link(s) which has one node before timestamp and one node after timestamp is kept.</p> <code>False</code> <code>exclude_links_to_past</code> <p>If True, links from nodes before <code>timestamp_past</code> to after <code>timestamp_past</code> are removed. If False, the link(s) which has one node before timestamp and one node after timestamp is kept.</p> <code>False</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary describing the lineage graph and operations for the root ID. Not returned if <code>as_nx_graph</code> is True. The dictionary contains the following keys:</p> <ul> <li>\"directed\" : bool<ul> <li>Whether the graph is directed.</li> </ul> </li> <li>\"graph\" : dict<ul> <li>Dictionary of graph attributes.</li> </ul> </li> <li>\"links\" : list of dict<ul> <li>Each element of the list is a dictionary describing an edge in the lineage graph as \"source\" and \"target\" keys.</li> </ul> </li> <li>\"multigraph\" : bool<ul> <li>Whether the graph is a multigraph.</li> </ul> </li> <li>\"nodes\" : list of dict<ul> <li>Each element of the list is a dictionary describing a node in the lineage graph, usually with \"id\", \"timestamp\", and \"operation_id\" keys.</li> </ul> </li> </ul> <code>DiGraph</code> <p>NetworkX directed graph of the lineage graph. Only returned if <code>as_nx_graph</code> is True.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_latest_roots","title":"<code>get_latest_roots(root_id, timestamp=None, timestamp_future=None)</code>","text":"<p>Returns root IDs that are related to the given <code>root_id</code> at a given timestamp. Can be used to find the \"latest\" root IDs associated with an object.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID.</p> required <code>timestamp</code> <code>datetime or None</code> <p>Timestamp of where to query IDs from. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <code>timestamp_future</code> <code>datetime or None</code> <p>DEPRECATED name, use <code>timestamp</code> instead. Timestamp to suggest IDs from (note can be in the past relative to the root). By default, None.</p> <code>None</code> <p>Returns:</p> Type Description <code>ndarray</code> <p>1d array with all latest successors.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_original_roots","title":"<code>get_original_roots(root_id, timestamp_past=None)</code>","text":"<p>Returns root IDs that are the latest successors of a given root ID.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID.</p> required <code>timestamp_past</code> <code>datetime or None</code> <p>Cutoff for the search going backwards in time. By default, None.</p> <code>None</code> <p>Returns:</p> Type Description <code>ndarray</code> <p>1d array with all latest successors.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.is_latest_roots","title":"<code>is_latest_roots(root_ids, timestamp=None)</code>","text":"<p>Check whether these root IDs are still a root at this timestamp.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>array-like of int</code> <p>Root IDs to check.</p> required <code>timestamp</code> <code>datetime</code> <p>Timestamp to check whether these IDs are valid root IDs in the chunked graph. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of bool</code> <p>Array of whether these are valid root IDs.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.suggest_latest_roots","title":"<code>suggest_latest_roots(root_id, timestamp=None, stop_layer=None, return_all=False, return_fraction_overlap=False)</code>","text":"<p>Suggest latest roots for a given root id, based on overlap of component chunk IDs. Note that edits change chunk IDs, and so this effectively measures the fraction of unchanged chunks at a given chunk layer, which sets the size scale of chunks. Higher layers are coarser.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID of the potentially outdated object.</p> required <code>timestamp</code> <code>datetime</code> <p>Datetime at which \"latest\" roots are being computed, by default None. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time. Note that this has to be a timestamp after the creation of the <code>root_id</code>.</p> <code>None</code> <code>stop_layer</code> <code>int</code> <p>Chunk level at which to compute overlap, by default None. No value will take the 4th from the top layer, which emphasizes speed and works well for larger objects. Lower values are slower but more fine-grained. Values under 2 (i.e. supervoxels) are not recommended except in extremely fine grained scenarios.</p> <code>None</code> <code>return_all</code> <code>bool</code> <p>If True, return all current IDs sorted from most overlap to least, by default False. If False, only the top is returned.</p> <code>False</code> <code>return_fraction_overlap</code> <code>bool</code> <p>If True, return all fractions sorted by most overlap to least, by default False. If False, only the top value is returned.</p> <code>False</code>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.is_valid_nodes","title":"<code>is_valid_nodes(node_ids, start_timestamp=None, end_timestamp=None)</code>","text":"<p>Check whether nodes are valid for given timestamp range.</p> <p>Valid is defined as existing in the chunked graph. This makes no statement about these IDs being roots, supervoxel or anything in-between. It also does not take into account whether a root ID has since been edited.</p> <p>Parameters:</p> Name Type Description Default <code>node_ids</code> <code>array-like of int</code> <p>Node IDs to check.</p> required <code>start_timestamp</code> <code>datetime</code> <p>Timestamp to check whether these IDs were valid after this timestamp. Defaults to None (assumes now).</p> <code>None</code> <code>end_timestamp</code> <code>datetime</code> <p>Timestamp to check whether these IDs were valid before this timestamp. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of bool</code> <p>Array of whether these are valid IDs.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_root_timestamps","title":"<code>get_root_timestamps(root_ids, latest=False, timestamp=None)</code>","text":"<p>Retrieves timestamps when roots where created.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <p>Iterable of root IDs to query.</p> required <code>latest</code> <code>bool</code> <p>If False,  returns the first timestamp that the root_id was valid for each root ID. If True, returns the newest/latest timestamp for each root ID. Note, this will return the timestamp at which the query was run when the root is currently valid. This means that you will get a different answer if you make this same query at a later time if you don't specify a timestamp parameter.</p> <code>False</code> <code>timestamp</code> <code>datetime</code> <p>Timestamp to query when using latest=True. Use this to provide consistent results for a particular timestamp. If an ID is still valid at a point in the future past this timestamp, the query will still return this timestamp as the latest moment in time. An error will occur if you provide a timestamp for which the root ID is not valid. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of datetime.datetime</code> <p>Array of timestamps when <code>root_ids</code> were created.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_past_ids","title":"<code>get_past_ids(root_ids, timestamp_past=None, timestamp_future=None)</code>","text":"<p>For a set of root IDs, get the list of IDs at a past or future time point that could contain parts of the same object.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>Iterable of int</code> <p>Iterable of root IDs to query.</p> required <code>timestamp_past</code> <code>datetime or None</code> <p>Time of a point in the past for which to look up root ids. Default is None.</p> <code>None</code> <code>timestamp_future</code> <code>datetime or None</code> <p>Time of a point in the future for which to look up root ids. Not implemented on the server currently. Default is None.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dict with keys \"future_id_map\" and \"past_id_map\". Each is a dict whose keys are the supplied <code>root_ids</code> and whose values are the list of related root IDs at <code>timestamp_past</code>/<code>timestamp_future</code>.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_delta_roots","title":"<code>get_delta_roots(timestamp_past, timestamp_future=datetime.datetime.now(datetime.timezone.utc))</code>","text":"<p>Get the list of roots that have changed between <code>timetamp_past</code> and <code>timestamp_future</code>.</p> <p>Parameters:</p> Name Type Description Default <code>timestamp_past</code> <code>datetime</code> <p>Past timepoint to query</p> required <code>timestamp_future</code> <code>datetime</code> <p>Future timepoint to query. Defaults to <code>datetime.datetime.now(datetime.timezone.utc)</code>.</p> <code>now(utc)</code> <p>Returns:</p> Name Type Description <code>old_roots</code> <code>np.ndarray of np.int64</code> <p>Roots that have expired in that interval.</p> <code>new_roots</code> <code>np.ndarray of np.int64</code> <p>Roots that are new in that interval.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_oldest_timestamp","title":"<code>get_oldest_timestamp()</code>","text":"<p>Get the oldest timestamp in the database.</p> <p>Returns:</p> Type Description <code>datetime</code> <p>Oldest timestamp in the database.</p>"},{"location":"api/client/","title":"caveclient.CAVEclient","text":"<p>A manager for all clients sharing common datastack and authentication information.</p> <p>This client wraps all the other clients and keeps track of the things that need to be consistent across them. To instantiate a client:</p> <pre><code>from caveclient import CAVEclient\n\nclient = CAVEclient(datastack_name='my_datastack',\n                    server_address='www.myserver.com',\n                    auth_token_file='~/.mysecrets/secrets.json')\n</code></pre> <p>Then</p> <ul> <li><code>client.annotation</code> is an <code>AnnotationClient</code> (see client.annotation)</li> <li><code>client.auth</code> is an <code>AuthClient</code> (see client.auth)</li> <li><code>client.chunkedgraph</code> is a <code>ChunkedGraphClient</code> (see client.chunkedgraph)</li> <li><code>client.info</code> is an <code>InfoServiceClient</code> (see client.info)</li> <li><code>client.l2cache</code> is an <code>L2CacheClient</code> (see client.l2cache)</li> <li><code>client.materialize</code> is a <code>MaterializationClient</code> (see client.materialize)</li> <li><code>client.skeleton</code> is a <code>SkeletonClient</code> (see client.skeleton)</li> <li><code>client.schema</code> is a <code>SchemaClient</code> (see client.schema)</li> <li><code>client.state</code> is a neuroglancer <code>JSONService</code> (see client.state)</li> </ul> <p>All subclients are loaded lazily and share the same datastack name, server address, and auth tokens where used.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>Datastack name for the services. Almost all services need this and will not work if it is not passed.</p> <code>None</code> <code>server_address</code> <code>str or None</code> <p>URL of the framework server. If None, chooses the default server global.daf-apis.com. Optional, defaults to None.</p> <code>None</code> <code>auth_token_file</code> <code>str or None</code> <p>Path to a json file containing the auth token. If None, uses the default location. See Auth client documentation. Optional, defaults to None.</p> <code>default_token_file</code> <code>auth_token_key</code> <code>str</code> <p>Dictionary key for the token in the the JSON file. Optional, default is 'token'.</p> <code>'token'</code> <code>auth_token</code> <code>str or None</code> <p>Direct entry of an auth token. If None, uses the file arguments to find the token. Optional, default is None.</p> <code>None</code> <code>max_retries</code> <code>int or None</code> <p>Sets the default number of retries on failed requests. Optional, by default 2.</p> <code>None</code> <code>pool_maxsize</code> <code>int or None</code> <p>Sets the max number of threads in a requests pool, although this value will be exceeded if pool_block is set to False. Optional, uses requests defaults if None.</p> <code>None</code> <code>pool_block</code> <p>If True, prevents the number of threads in a requests pool from exceeding the max size. Optional, uses requests defaults (False) if None.</p> <code>None</code> <code>desired_resolution</code> <code>Iterable[float] or None</code> <p>If given, should be a list or array of the desired resolution you want queries returned in useful for materialization queries.</p> <code>None</code> <code>info_cache</code> <p>Pre-computed info cache, bypassing the lookup of datastack info from the info service. Should only be used in cases where this information is cached and thus repetitive lookups can be avoided.</p> <code>None</code> <code>version</code> <code>Optional[int]</code> <p>The default materialization version of the datastack to use. If None, the latest version is used. Optional, defaults to None.</p> <code>None</code> See Also <p>set_session_defaults</p> <p>get_session_defaults</p> <p>Methods:</p> Name Description <code>change_auth</code> <p>Change the authentication token and reset services.</p> <p>Attributes:</p> Name Type Description <code>annotation</code> <code>AnnotationClient</code> <p>A client for the annotation service. See client.annotation</p> <code>auth</code> <code>AuthClient</code> <p>A client for the auth service. See client.auth for more information.</p> <code>chunkedgraph</code> <code>ChunkedGraphClient</code> <p>A client for the chunkedgraph service. See client.chunkedgraph</p> <code>datastack_name</code> <code>str</code> <p>The name of the datastack for the client.</p> <code>info</code> <code>InfoServiceClient</code> <p>A client for the info service. See client.info for more information.</p> <code>l2cache</code> <code>L2CacheClient</code> <p>A client for the L2 cache service. See client.l2cache</p> <code>materialize</code> <code>MaterializationClient</code> <p>A client for the materialization service. See client.materialize</p> <code>schema</code> <code>SchemaClient</code> <p>A client for the EM Annotation Schemas service. See client.schema</p> <code>server_address</code> <p>The server address for the client.</p> <code>skeleton</code> <code>SkeletonClient</code> <p>A client for the skeleton service. See client.skeleton</p> <code>state</code> <code>JSONService</code> <p>A client for the neuroglancer state service. See client.state</p> <code>timestamp</code> <code>Optional[datetime]</code> <p>The default timestamp to use for queries which rely on a timestamp.</p> <code>version</code> <code>Optional[int]</code> <p>The default materialization version of the datastack to use for queries which</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.annotation","title":"<code>annotation: AnnotationClient</code>  <code>property</code>","text":"<p>A client for the annotation service. See client.annotation for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.auth","title":"<code>auth: AuthClient</code>  <code>property</code>","text":"<p>A client for the auth service. See client.auth for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.chunkedgraph","title":"<code>chunkedgraph: ChunkedGraphClient</code>  <code>property</code>","text":"<p>A client for the chunkedgraph service. See client.chunkedgraph for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.datastack_name","title":"<code>datastack_name: str</code>  <code>property</code>","text":"<p>The name of the datastack for the client.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.info","title":"<code>info: InfoServiceClient</code>  <code>property</code>","text":"<p>A client for the info service. See client.info for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.l2cache","title":"<code>l2cache: L2CacheClient</code>  <code>property</code>","text":"<p>A client for the L2 cache service. See client.l2cache for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.materialize","title":"<code>materialize: MaterializationClient</code>  <code>property</code>","text":"<p>A client for the materialization service. See client.materialize for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.schema","title":"<code>schema: SchemaClient</code>  <code>property</code>","text":"<p>A client for the EM Annotation Schemas service. See client.schema for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.server_address","title":"<code>server_address</code>  <code>property</code>","text":"<p>The server address for the client.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.skeleton","title":"<code>skeleton: SkeletonClient</code>  <code>property</code>","text":"<p>A client for the skeleton service. See client.skeleton for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.state","title":"<code>state: JSONService</code>  <code>property</code>","text":"<p>A client for the neuroglancer state service. See client.state for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.timestamp","title":"<code>timestamp: Optional[datetime]</code>  <code>property</code>","text":"<p>The default timestamp to use for queries which rely on a timestamp.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.version","title":"<code>version: Optional[int]</code>  <code>property</code> <code>writable</code>","text":"<p>The default materialization version of the datastack to use for queries which expect a version. Also sets the timestamp to the corresponding timestamp of the version for queries which rely on a timestamp.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.change_auth","title":"<code>change_auth(auth_token_file=None, auth_token_key=None, auth_token=None)</code>","text":"<p>Change the authentication token and reset services.</p> <p>Parameters:</p> Name Type Description Default <code>auth_token_file</code> <code>str</code> <p>New auth token json file path, by default None, which defaults to the existing state.</p> <code>None</code> <code>auth_token_key</code> <code>str</code> <p>New dictionary key under which the token is stored in the json file, by default None, which defaults to the existing state.</p> <code>None</code> <code>auth_token</code> <code>str</code> <p>Direct entry of a new token, by default None.</p> <code>None</code>"},{"location":"api/config/","title":"Configuration","text":""},{"location":"api/config/#caveclient.set_session_defaults","title":"<code>caveclient.set_session_defaults(max_retries=3, pool_block=False, pool_maxsize=10, backoff_factor=0.1, backoff_max=120, status_forcelist=(502, 503, 504))</code>","text":"<p>Set global default values to configure how all clients will communicate with servers. Should be done prior to initializing a client.</p> <p>Note that these values are only used when not set at the client level.</p> <p>Parameters:</p> Name Type Description Default <code>max_retries</code> <code>int</code> <p>The maximum number of retries each connection should attempt. Set to 0 to fail on the first retry.</p> <code>3</code> <code>pool_block</code> <code>bool</code> <p>Whether the connection pool should block for connections.</p> <code>False</code> <code>pool_maxsize</code> <code>int</code> <p>The maximum number of connections to save in the pool.</p> <code>10</code> <code>backoff_factor</code> <code>Union[float, int]</code> <p>A backoff factor to apply between attempts after the second try (most errors are resolved immediately by a second try without a delay). The query will sleep for:     <code>{backoff factor} * (2 ^ ({number of total retries} - 1))</code> seconds. For example, if the <code>backoff_factor</code> is 0.1, then will sleep for [0.0s, 0.2s, 0.4s, 0.8s, \u2026] between retries. No backoff will ever be longer than <code>backoff_max</code>.</p> <code>0.1</code> <code>backoff_max</code> <code>Union[float, int]</code> <p>The maximum backoff time.</p> <code>120</code> <code>status_forcelist</code> <code>Optional[Collection]</code> <p>A set of integer HTTP status codes that we should force a retry on.</p> <code>(502, 503, 504)</code> Usage <pre><code>from caveclient import set_session_defaults\n\nset_session_defaults(\n    max_retries=5, # would increase the default number of retries\n    backoff_factor=0.5, # would increase the default backoff factor between retries\n    backoff_max=240, # would increase the default maximum backoff time\n    status_forcelist=(502, 503, 504, 505), # would add 505 to the default list\n)\n\nset_session_defaults() # would revert all defaults to their original values\n</code></pre> Notes <p>Calling this function will set the default values for all clients created after the call.</p> <p>Calling this function with any arguments missing will reset that value to the default value.</p> See Also: <p>urllib3.util.Retry</p> <p>requests.adapters.HTTPAdapter</p>"},{"location":"api/config/#caveclient.get_session_defaults","title":"<code>caveclient.get_session_defaults()</code>","text":"<p>Get the current default values for session configuration.</p> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of current default values for session configuration.</p>"},{"location":"api/datastack_lookup/","title":"Datastack lookup","text":""},{"location":"api/datastack_lookup/#caveclient.datastack_lookup.reset_server_address_cache","title":"<code>caveclient.datastack_lookup.reset_server_address_cache(datastack, filename=None)</code>","text":"<p>Remove one or more datastacks from the datastack-to-server cache.</p> <p>Parameters:</p> Name Type Description Default <code>datastack</code> <code>str or list of str</code> <p>Datastack names to remove from the cache, by default None</p> required <code>filename</code> <code>str</code> <p>Name of the cache file, by default None</p> <code>None</code>"},{"location":"api/info/","title":"client.info","text":"<p>Client for interacting with the info service.</p> <p>Methods:</p> Name Description <code>annotation_endpoint</code> <p>AnnotationEngine endpoint for a dataset.</p> <code>get_aligned_volume_info</code> <p>Gets the info record for a aligned_volume</p> <code>get_datastack_info</code> <p>Gets the info record for a datastack</p> <code>get_datastacks</code> <p>Query which datastacks are available at the info service</p> <code>get_datastacks_by_aligned_volume</code> <p>Lookup what datastacks are associated with this aligned volume</p> <code>image_cloudvolume</code> <p>Generate a cloudvolume instance based on the image source, using authentication if needed and</p> <code>image_source</code> <p>Cloud path to the imagery for the dataset</p> <code>refresh_stored_data</code> <p>Reload the stored info values from the server.</p> <code>segmentation_cloudvolume</code> <p>Generate a cloudvolume instance based on the segmentation source, using authentication if needed and</p> <code>segmentation_source</code> <p>Cloud path to the chunkgraph-backed Graphene segmentation for a dataset</p> <code>synapse_segmentation_source</code> <p>Cloud path to the synapse segmentation for a dataset</p> <code>viewer_resolution</code> <p>Get the viewer resolution metadata for this datastack</p> <code>viewer_site</code> <p>Get the base Neuroglancer URL for the dataset</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.annotation_endpoint","title":"<code>annotation_endpoint(datastack_name=None, use_stored=True)</code>","text":"<p>AnnotationEngine endpoint for a dataset.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <p>Returns:</p> Type Description <code>str</code> <p>Location of the AnnotationEngine</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_aligned_volume_info","title":"<code>get_aligned_volume_info(datastack_name=None, use_stored=True)</code>","text":"<p>Gets the info record for a aligned_volume</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>datastack_name to look up. If None, uses the one specified by the client. By default None</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True and the information has already been queried for that dataset, then uses the cached version. If False, re-queries the infromation. By default True</p> <code>True</code> <p>Returns:</p> Type Description <code>dict or None</code> <p>The complete info record for the aligned_volume</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_datastack_info","title":"<code>get_datastack_info(datastack_name=None, use_stored=True)</code>","text":"<p>Gets the info record for a datastack</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>datastack to look up. If None, uses the one specified by the client. By default None</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True and the information has already been queried for that datastack, then uses the cached version. If False, re-queries the infromation. By default True</p> <code>True</code> <p>Returns:</p> Type Description <code>dict or None</code> <p>The complete info record for the datastack</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_datastacks","title":"<code>get_datastacks()</code>","text":"<p>Query which datastacks are available at the info service</p> <p>Returns:</p> Type Description <code>list</code> <p>List of datastack names</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_datastacks_by_aligned_volume","title":"<code>get_datastacks_by_aligned_volume(aligned_volume=None)</code>","text":"<p>Lookup what datastacks are associated with this aligned volume</p> <p>Args:     aligned_volume (str, optional): aligned volume to lookup. Defaults to None.</p> <p>Raises:     ValueError: if no aligned volume is specified</p> <p>Returns:     list: a list of datastack string</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.image_cloudvolume","title":"<code>image_cloudvolume(**kwargs)</code>","text":"<p>Generate a cloudvolume instance based on the image source, using authentication if needed and sensible default values for reading CAVE resources. By default, fill_missing is True and bounded is False. All keyword arguments are passed onto the CloudVolume initialization function, and defaults can be overridden.</p> <p>Requires cloudvolume to be installed, which is not included by default.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.image_source","title":"<code>image_source(datastack_name=None, use_stored=True, format_for='raw')</code>","text":"<p>Cloud path to the imagery for the dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <code>format_for</code> <code>'raw', 'cloudvolume', or 'neuroglancer'</code> <p>Formats the path for different uses. If 'raw' (default), the path in the InfoService is passed along. If 'cloudvolume', a \"precomputed://gs://\" type path is converted to a full https URL. If 'neuroglancer', a full https URL is converted to a \"precomputed://gs://\" type path.</p> <code>'raw'</code> <p>Returns:</p> Type Description <code>str</code> <p>Formatted cloud path to the flat segmentation</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.refresh_stored_data","title":"<code>refresh_stored_data()</code>","text":"<p>Reload the stored info values from the server.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.segmentation_cloudvolume","title":"<code>segmentation_cloudvolume(use_client_secret=True, **kwargs)</code>","text":"<p>Generate a cloudvolume instance based on the segmentation source, using authentication if needed and sensible default values for reading CAVE resources. By default, fill_missing is True and bounded is False. All keyword arguments are passed onto the CloudVolume initialization function, and defaults can be overridden.</p> <p>Requires cloudvolume to be installed, which is not included by default.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.segmentation_source","title":"<code>segmentation_source(datastack_name=None, format_for='raw', use_stored=True)</code>","text":"<p>Cloud path to the chunkgraph-backed Graphene segmentation for a dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <code>format_for</code> <code>'raw', 'cloudvolume', or 'neuroglancer'</code> <p>Formats the path for different uses. If 'raw' (default), the path in the InfoService is passed along. If 'cloudvolume', a \"graphene://https://\" type path is used If 'neuroglancer', a \"graphene://https://\" type path is used, as needed by Neuroglancer.</p> <code>'raw'</code> <p>Returns:</p> Type Description <code>str</code> <p>Formatted cloud path to the Graphene segmentation</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.synapse_segmentation_source","title":"<code>synapse_segmentation_source(datastack_name=None, use_stored=True, format_for='raw')</code>","text":"<p>Cloud path to the synapse segmentation for a dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the dataset to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <code>format_for</code> <code>'raw', 'cloudvolume', or 'neuroglancer'</code> <p>Formats the path for different uses. If 'raw' (default), the path in the InfoService is passed along. If 'cloudvolume', a \"precomputed://gs://\" type path is converted to a full https URL. If 'neuroglancer', a full https URL is converted to a \"precomputed://gs://\" type path.</p> <code>'raw'</code> <p>Returns:</p> Type Description <code>str</code> <p>Formatted cloud path to the synapse segmentation</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.viewer_resolution","title":"<code>viewer_resolution(datastack_name=None, use_stored=True)</code>","text":"<p>Get the viewer resolution metadata for this datastack</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <p>If None use the default one configured in the client</p> <code>None</code> <code>use_stored</code> <p>Use the cached value, if False go get a new value from server</p> <code>True</code> <p>Returns:</p> Type Description <code>array</code> <p>Voxel resolution as a len(3) np.array</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.viewer_site","title":"<code>viewer_site(datastack_name=None, use_stored=True)</code>","text":"<p>Get the base Neuroglancer URL for the dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <p>Returns:</p> Type Description <code>str</code> <p>Base URL for the Neuroglancer viewer</p>"},{"location":"api/l2cache/","title":"client.l2cache","text":"<p>Client for interacting with the level2 cache service.</p> <p>Methods:</p> Name Description <code>cache_metadata</code> <p>Retrieves the meta data for the cache</p> <code>get_l2data</code> <p>Gets the attributed statistics data for L2 ids.</p> <code>get_l2data_table</code> <p>Gets the attributed statistics data for L2 ids, returned as a dataframe.</p> <code>has_cache</code> <p>Checks if the l2 cache is available for the dataset</p> <code>table_mapping</code> <p>Retrieves table mappings for l2 cache.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.cache_metadata","title":"<code>cache_metadata()</code>","text":"<p>Retrieves the meta data for the cache</p> <p>Returns:</p> Type Description <code>dict</code> <p>keys are attribute names, values are datatypes</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.get_l2data","title":"<code>get_l2data(l2_ids, attributes=None)</code>","text":"<p>Gets the attributed statistics data for L2 ids.</p> <p>Parameters:</p> Name Type Description Default <code>l2_ids</code> <code>list or ndarray</code> <p>A list of level 2 ids.</p> required <code>attributes</code> <code>list</code> <p>A list of attributes to retrieve. Defaults to <code>None</code>, which will return all that are available. Available stats are:</p> <ul> <li><code>area_nm2</code></li> <li><code>chunk_intersect_count</code></li> <li><code>max_dt_nm</code></li> <li><code>mean_dt_nm</code></li> <li><code>pca</code></li> <li><code>pca_val</code></li> <li><code>rep_coord_nm</code></li> <li><code>size_nm3</code></li> </ul> <p>See the tutorial for more description.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Keys are l2 ids, values are data.</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.get_l2data_table","title":"<code>get_l2data_table(l2_ids, attributes=None, split_columns=True)</code>","text":"<p>Gets the attributed statistics data for L2 ids, returned as a dataframe.</p> <p>Parameters:</p> Name Type Description Default <code>l2_ids</code> <code>list or ndarray</code> <p>A list of level 2 ids.</p> required <code>attributes</code> <code>list</code> <p>A list of attributes to retrieve. Defaults to <code>None</code>, which will return all that are available. Available stats are:</p> <ul> <li><code>area_nm2</code></li> <li><code>chunk_intersect_count</code></li> <li><code>max_dt_nm</code></li> <li><code>mean_dt_nm</code></li> <li><code>pca</code></li> <li><code>pca_val</code></li> <li><code>rep_coord_nm</code></li> <li><code>size_nm3</code></li> </ul> <p>See the tutorial for more description.</p> <code>None</code> <code>split_columns</code> <code>bool</code> <p>Whether to split columns with multiple values into separate columns.</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe with the requested attributes as columns and indexed by l2_id.</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.has_cache","title":"<code>has_cache(datastack_name=None)</code>","text":"<p>Checks if the l2 cache is available for the dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check, by default None (if None, uses the client's datastack)</p> <code>None</code> <p>Returns:</p> Type Description <code>bool</code> <p>True if the l2 cache is available, False otherwise</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.table_mapping","title":"<code>table_mapping()</code>","text":"<p>Retrieves table mappings for l2 cache.</p> <p>Returns:</p> Type Description <code>dict</code> <p>keys are pcg table names, values are dicts with fields <code>l2cache_id</code> and <code>cv_path</code>.</p>"},{"location":"api/materialize/","title":"client.materialize","text":"<p>Client for interacting with the materialization engine.</p> <p>Methods:</p> Name Description <code>get_table_metadata</code> <p>Get metadata about a table</p> <code>get_tables</code> <p>Gets a list of table names for a datastack</p> <code>get_tables_metadata</code> <p>Get metadata about tables</p> <code>get_timestamp</code> <p>Get datetime.datetime timestamp for a materialization version.</p> <code>get_unique_string_values</code> <p>Get unique string values for a table</p> <code>get_version_metadata</code> <p>Get metadata about a version</p> <code>get_versions</code> <p>Get the versions available</p> <code>get_versions_metadata</code> <p>Get the metadata for all the versions that are presently available and valid</p> <code>get_view_metadata</code> <p>Get metadata for a view</p> <code>get_view_schema</code> <p>Get schema for a view</p> <code>get_view_schemas</code> <p>Get schema for a view</p> <code>get_views</code> <p>Get all available views for a version</p> <code>ingest_annotation_table</code> <p>Trigger supervoxel lookup and root ID lookup of new annotations in a table.</p> <code>join_query</code> <p>Generic query on materialization tables</p> <code>live_live_query</code> <p>Beta method for querying cave annotation tables with root IDs and annotations</p> <code>live_query</code> <p>Generic query on materialization tables</p> <code>lookup_supervoxel_ids</code> <p>Trigger supervoxel lookups of new annotations in a table.</p> <code>map_filters</code> <p>Translate a list of filter dictionaries from a point in the</p> <code>most_recent_version</code> <p>Get the most recent version of materialization for this datastack name</p> <code>query_table</code> <p>Generic query on materialization tables</p> <code>query_view</code> <p>Generic query on a view</p> <code>synapse_query</code> <p>Convenience method for querying synapses.</p> <p>Attributes:</p> Name Type Description <code>cg_client</code> <p>The chunked graph client.</p> <code>datastack_name</code> <p>The name of the datastack.</p> <code>homepage</code> <code>HTML</code> <p>The homepage for the materialization engine.</p> <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p> <code>tables</code> <code>TableManager</code> <p>The table manager for the materialization engine.</p> <code>version</code> <code>int</code> <p>The version of the materialization. Can be used to set up the</p> <code>views</code> <code>ViewManager</code> <p>The view manager for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.cg_client","title":"<code>cg_client</code>  <code>property</code>","text":"<p>The chunked graph client.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.datastack_name","title":"<code>datastack_name</code>  <code>property</code>","text":"<p>The name of the datastack.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.homepage","title":"<code>homepage: HTML</code>  <code>property</code>","text":"<p>The homepage for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.tables","title":"<code>tables: TableManager</code>  <code>property</code>","text":"<p>The table manager for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.version","title":"<code>version: int</code>  <code>property</code> <code>writable</code>","text":"<p>The version of the materialization. Can be used to set up the client to default to a specific version when timestamps or versions are not specified in queries. If not set, defaults to the most recent version.</p> <p>Note that if this materialization client is attached to a CAVEclient, the version must be set at the CAVEclient level.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.views","title":"<code>views: ViewManager</code>  <code>property</code>","text":"<p>The view manager for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_table_metadata","title":"<code>get_table_metadata(table_name, datastack_name=None, version=None, log_warning=True)</code>","text":"<p>Get metadata about a table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack_name. If None, uses the one specified in the client.</p> <code>None</code> <code>version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to print out warnings to the logger. Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Metadata dictionary for table</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_tables","title":"<code>get_tables(datastack_name=None, version=None)</code>","text":"<p>Gets a list of table names for a datastack</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client</p> <code>None</code> <code>version</code> <code>int or None</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>List of table names</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_tables_metadata","title":"<code>get_tables_metadata(datastack_name=None, version=None, log_warning=True)</code>","text":"<p>Get metadata about tables</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack_name. If None, uses the one specified in the client.</p> <code>None</code> <code>version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to print out warnings to the logger. Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Metadata dictionary for table</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_timestamp","title":"<code>get_timestamp(version=None, datastack_name=None)</code>","text":"<p>Get datetime.datetime timestamp for a materialization version.</p> <p>Parameters:</p> Name Type Description Default <code>version</code> <code>int or None</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>datastack_name</code> <code>str or None</code> <p>Datastack name, by default None. If None, defaults to the value set in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>datetime</code> <p>Datetime when the materialization version was frozen.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_unique_string_values","title":"<code>get_unique_string_values(table, datastack_name=None)</code>","text":"<p>Get unique string values for a table</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Table to query</p> required <code>datastack_name</code> <code>Optional[str]</code> <p>Datastack to query. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict[str]</code> <p>A dictionary of column names and their unique values</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_version_metadata","title":"<code>get_version_metadata(version=None, datastack_name=None)</code>","text":"<p>Get metadata about a version</p> <p>Parameters:</p> Name Type Description Default <code>version</code> <code>int or None</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>datastack_name</code> <code>str or None</code> <p>Datastack name, by default None. If None, defaults to the value set in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of metadata about the version</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_versions","title":"<code>get_versions(datastack_name=None, expired=False)</code>","text":"<p>Get the versions available</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack, by default None. If None, uses the one specified in the client.</p> <code>None</code> <code>expired</code> <code>bool</code> <p>Whether to include expired versions, by default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of versions available</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_versions_metadata","title":"<code>get_versions_metadata(datastack_name=None, expired=False)</code>","text":"<p>Get the metadata for all the versions that are presently available and valid</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Datastack name, by default None. If None, defaults to the value set in the client.</p> <code>None</code> <code>expired</code> <code>bool</code> <p>Whether to include expired versions, by default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>list[dict]</code> <p>List of metadata dictionaries</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_view_metadata","title":"<code>get_view_metadata(view_name, materialization_version=None, datastack_name=None, log_warning=True)</code>","text":"<p>Get metadata for a view</p> <p>Parameters:</p> Name Type Description Default <code>view_name</code> <code>str</code> <p>Name of view to query.</p> required <code>materialization_version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Metadata of view</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_view_schema","title":"<code>get_view_schema(view_name, materialization_version=None, datastack_name=None, log_warning=True)</code>","text":"<p>Get schema for a view</p> <p>Parameters:</p> Name Type Description Default <code>view_name</code> <code>str</code> <p>Name of view to query.</p> required <code>materialization_version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Schema of view.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_view_schemas","title":"<code>get_view_schemas(materialization_version=None, datastack_name=None, log_warning=True)</code>","text":"<p>Get schema for a view</p> <p>Parameters:</p> Name Type Description Default <code>materialization_version</code> <code>Optional[int]</code> <p>Version to query. If None, will use version set by client.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Schema of view.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_views","title":"<code>get_views(version=None, datastack_name=None)</code>","text":"<p>Get all available views for a version</p> <p>Parameters:</p> Name Type Description Default <code>version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>List of views</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.ingest_annotation_table","title":"<code>ingest_annotation_table(table_name, datastack_name=None)</code>","text":"<p>Trigger supervoxel lookup and root ID lookup of new annotations in a table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Table to trigger</p> required <code>datastack_name</code> <code>str</code> <p>Datastack to trigger it. Defaults to what is set in client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Status code of response from server</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.join_query","title":"<code>join_query(tables, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, suffixes=None, datastack_name=None, return_df=True, split_positions=False, materialization_version=None, metadata=True, desired_resolution=None, random_sample=None, log_warning=True)</code>","text":"<p>Generic query on materialization tables</p> <p>Parameters:</p> Name Type Description Default <code>tables</code> <code>list of lists with length 2 or 'str'</code> <p>list of two lists: first entries are table names, second entries are the columns used for the join.</p> required <code>filter_in_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are allowed entries, by default None</p> <code>None</code> <code>filter_out_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are not allowed entries, by default None</p> <code>None</code> <code>filter_equal_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are specified entry, by default None</p> <code>None</code> <code>filter_greater_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are exclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are exclusive lower-bound, by default None</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are inclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are inclusive lower-bound, by default None</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names, inner layer: keys are column names. Values are bounding boxes as [[min_x, min_y,min_z],[max_x, max_y, max_z]], expressed in units of the voxel_resolution of this dataset. Defaults to None.</p> <code>None</code> <code>filter_regex_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names. inner layer: keys are column names, values are regex strings. Defaults to None</p> <code>None</code> <code>select_columns</code> <code>dict of lists of str</code> <p>keys are table names,values are the list of columns from that table. Defaults to None, which will select all tables.  Will be passed to server as select_column_maps. Passing a list will be passed as select_columns which is deprecated.</p> <code>None</code> <code>offset</code> <code>int</code> <p>result offset to use. Defaults to None. Will only return top K results.</p> <code>None</code> <code>limit</code> <code>int</code> <p>maximum results to return (server will set upper limit, see get_server_config)</p> <code>None</code> <code>suffixes</code> <code>dict</code> <p>suffixes to use for duplicate columns, keys are table names, values are the suffix</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>datastack to query. If None defaults to one specified in client.</p> <code>None</code> <code>return_df</code> <code>bool</code> <p>whether to return as a dataframe default True, if False, data is returned as json (slower)</p> <code>True</code> <code>split_positions</code> <code>bool</code> <p>whether to break position columns into x,y,z columns default False, if False data is returned as one column with [x,y,z] array (slower)</p> <code>False</code> <code>materialization_version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>toggle to return metadata If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>desired_resolution</code> <code>Iterable</code> <p>What resolution to convert position columns to. Defaults to None will use defaults.</p> <code>None</code> <code>random_sample</code> <code>int</code> <p>if given, will do a tablesample of the table to return that many annotations</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings, by default True</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>a pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.live_live_query","title":"<code>live_live_query(table, timestamp, joins=None, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, split_positions=False, metadata=True, suffixes=None, desired_resolution=None, allow_missing_lookups=False, allow_invalid_root_ids=False, random_sample=None, log_warning=True)</code>","text":"<p>Beta method for querying cave annotation tables with root IDs and annotations at a particular timestamp.  Note: this method requires more explicit mapping of filters and selection to table as its designed to test a more general endpoint that should eventually support complex joins.</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Principle table to query</p> required <code>timestamp</code> <code>datetime</code> <p>Timestamp to query</p> required <code>joins</code> <p>List of joins, where each join is a list of [table1,column1, table2, column2]</p> <code>None</code> <code>filter_in_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and list values to accept.</p> <code>None</code> <code>filter_out_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and list values to reject.</p> <code>None</code> <code>filter_equal_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values to equate.</p> <code>None</code> <code>filter_greater_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as exclusive upper-bound.</p> <code>None</code> <code>filter_less_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as exclusive lower-bound.</p> <code>None</code> <code>filter_greater_equal_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as inclusive upper-bound.</p> <code>None</code> <code>filter_less_equal_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as inclusive lower-bound.</p> <code>None</code> <code>filter_spatial_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values of 2x3 list of bounds.</p> <code>None</code> <code>filter_regex_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values of regex strings.</p> <code>None</code> <code>select_columns</code> <p>A dictionary with tables as keys, values are lists of columns to select.</p> <code>None</code> <code>offset</code> <code>int</code> <p>Value to offset query by.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Limit of query.</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query. Defaults to set by client.</p> <code>None</code> <code>split_positions</code> <code>bool</code> <p>Whether to split positions into separate columns, True is faster.</p> <code>False</code> <code>metadata</code> <code>bool</code> <p>Whether to attach metadata to dataframe.</p> <code>True</code> <code>suffixes</code> <code>dict</code> <p>What suffixes to use on joins, keys are table_names, values are suffixes.</p> <code>None</code> <code>desired_resolution</code> <code>Iterable</code> <p>What resolution to convert position columns to.</p> <code>None</code> <code>allow_missing_lookups</code> <code>bool</code> <p>If there are annotations without supervoxels and root IDs yet, allow results.</p> <code>False</code> <code>allow_invalid_root_ids</code> <code>bool</code> <p>If True, ignore root ids not valid at the given timestamp, otherwise raise an error.</p> <code>False</code> <code>random_sample</code> <code>int</code> <p>If given, will do a table sample of the table to return that many annotations.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <p>Results of query</p> <p>Examples:</p> <pre><code>&gt;&gt;&gt; from caveclient import CAVEclient\n&gt;&gt;&gt; client = CAVEclient('minnie65_public_v117')\n&gt;&gt;&gt; live_live_query(\"table_name\", datetime.datetime.now(datetime.timezone.utc),\n&gt;&gt;&gt;    joins=[[table_name, table_column, joined_table, joined_column],\n&gt;&gt;&gt;             [joined_table, joincol2, third_table, joincol_third]]\n&gt;&gt;&gt;    suffixes={\n&gt;&gt;&gt;        \"table_name\":\"suffix1\",\n&gt;&gt;&gt;        \"joined_table\":\"suffix2\",\n&gt;&gt;&gt;        \"third_table\":\"suffix3\"\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    select_columns= {\n&gt;&gt;&gt;        \"table_name\":[ \"column\",\"names\"],\n&gt;&gt;&gt;        \"joined_table\":[\"joined_colum\"]\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    filter_in_dict= {\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":[included,values]\n&gt;&gt;&gt;        }\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    filter_out_dict= {\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":[excluded,values]\n&gt;&gt;&gt;        }\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    filter_equal_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_greater_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_less_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_greater_equal_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_less_equal_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_spatial_dict\"= {\n&gt;&gt;&gt;        \"table_name\": {\n&gt;&gt;&gt;        \"column_name\": [[min_x, min_y, min_z], [max_x, max_y, max_z]]\n&gt;&gt;&gt;    }\n&gt;&gt;&gt;    filter_regex_dict\"= {\n&gt;&gt;&gt;        \"table_name\": {\n&gt;&gt;&gt;        \"column_name\": \"regex_string\"\n&gt;&gt;&gt;     }\n</code></pre>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.live_query","title":"<code>live_query(table, timestamp, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, split_positions=False, post_filter=True, metadata=True, merge_reference=True, desired_resolution=None, random_sample=None, log_warning=True)</code>","text":"<p>Generic query on materialization tables</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Table to query</p> required <code>timestamp</code> <code>datetime</code> <p>Time to materialize (in utc). Pass datetime.datetime.now(datetime.timezone.utc) for present time.</p> required <code>filter_in_dict</code> <code>dict</code> <p>Keys are column names, values are allowed entries.</p> <code>None</code> <code>filter_out_dict</code> <code>dict</code> <p>Keys are column names, values are not allowed entries.</p> <code>None</code> <code>filter_equal_dict</code> <code>dict</code> <p>Keys are column names, values are specified entry.</p> <code>None</code> <code>filter_greater_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive upper-bounds.</p> <code>None</code> <code>filter_less_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive lower-bounds.</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive upper-bounds.</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive lower-bounds.</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict</code> <p>Keys are column names, values are bounding boxes expressed in units of the voxel_resolution of this dataset. Bounding box is [[min_x, min_y,min_z],[max_x, max_y, max_z]].</p> <code>None</code> <code>filter_regex_dict</code> <code>dict</code> <p>Keys are column names, values are regex strings.</p> <code>None</code> <code>select_columns</code> <code>list of str</code> <p>Columns to select.</p> <code>None</code> <code>offset</code> <code>int</code> <p>Offset in query result.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Maximum results to return (server will set upper limit, see get_server_config).</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query. If None, defaults to one specified in client.</p> <code>None</code> <code>split_positions</code> <code>bool</code> <p>Whether to break position columns into x,y,z columns. If False data is returned as one column with [x,y,z] array (slower).</p> <code>False</code> <code>post_filter</code> <code>bool</code> <p>Whether to filter down the result based upon the filters specified. If False, it will return the query with present root_ids in the root_id columns, but the rows will reflect the filters translated into their past IDs. So if, for example, a cell had a false merger split off since the last materialization, those annotations on that incorrect portion of the cell will be included if this is False, but will be filtered down if this is True.</p> <code>True</code> <code>metadata</code> <code>bool</code> <p>Toggle to return metadata. If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>merge_reference</code> <code>bool</code> <p>Toggle to automatically join reference table. If True, metadata will be queries and if its a reference table it will perform a join on the reference table to return the rows of that table.</p> <code>True</code> <code>desired_resolution</code> <code>Iterable</code> <p>Desired resolution you want all spatial points returned in. If None, defaults to one specified in client, if that is None then points are returned as stored in the table and should be in the resolution specified in the table metadata.</p> <code>None</code> <code>random_sample</code> <code>int</code> <p>If given, will do a tablesample of the table to return that many annotations.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.lookup_supervoxel_ids","title":"<code>lookup_supervoxel_ids(table_name, annotation_ids=None, datastack_name=None)</code>","text":"<p>Trigger supervoxel lookups of new annotations in a table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Table to trigger</p> required <code>annotation_ids</code> <code>list</code> <p>List of annotation ids to lookup. Default is None, which will trigger lookup of entire table.</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to trigger it. Defaults to what is set in client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Status code of response from server</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.map_filters","title":"<code>map_filters(filters, timestamp, timestamp_past)</code>","text":"<p>Translate a list of filter dictionaries from a point in the future to a point in the past</p> <p>Parameters:</p> Name Type Description Default <code>filters</code> <code>list[dict]</code> <p>filter dictionaries with root_ids</p> required <code>timestamp</code> <code>datetime</code> <p>timestamp to query</p> required <code>timestamp_past</code> <code>datetime</code> <p>timestamp to query from</p> required <p>Returns:</p> Type Description <code>list[dict]</code> <p>filter dictionaries with past root_ids</p> <code>dict</code> <p>mapping of future root_ids to past root_ids</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.most_recent_version","title":"<code>most_recent_version(datastack_name=None)</code>","text":"<p>Get the most recent version of materialization for this datastack name</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client</p> <code>None</code> <p>Returns:</p> Type Description <code>int</code> <p>Most recent version of materialization for this datastack name</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.query_table","title":"<code>query_table(table, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, return_df=True, split_positions=False, materialization_version=None, timestamp=None, metadata=True, merge_reference=True, desired_resolution=None, get_counts=False, random_sample=None, log_warning=True)</code>","text":"<p>Generic query on materialization tables</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Table to query</p> required <code>filter_in_dict</code> <code>dict</code> <p>Keys are column names, values are allowed entries, by default None</p> <code>None</code> <code>filter_out_dict</code> <code>dict</code> <p>Keys are column names, values are not allowed entries, by default None</p> <code>None</code> <code>filter_equal_dict</code> <code>dict</code> <p>Keys are column names, values are specified entry, by default None</p> <code>None</code> <code>filter_greater_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive upper-bound value, by default None</p> <code>None</code> <code>filter_less_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive lower-bound value, by default None</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive upper-bound value, by default None</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive lower-bound value, by default None</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict</code> <p>Keys are column names, values are bounding boxes expressed in units of the voxel_resolution of this dataset. Bounding box is [[min_x, min_y,min_z],[max_x, max_y, max_z]], by default None</p> <code>None</code> <code>filter_regex_dict</code> <code>dict</code> <p>Keys are column names, values are regex strings, by default None</p> <code>None</code> <code>select_columns</code> <code>list of str</code> <p>Columns to select, by default None</p> <code>None</code> <code>offset</code> <code>int</code> <p>Result offset to use, by default None. Will only return top K results.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Maximum results to return (server will set upper limit, see get_server_config), by default None</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query, by default None. If None, defaults to one specified in client.</p> <code>None</code> <code>return_df</code> <code>bool</code> <p>Whether to return as a dataframe, by default True. If False, data is returned as json (slower).</p> <code>True</code> <code>split_positions</code> <code>bool</code> <p>Whether to break position columns into x,y,z columns, by default False. If False data is returned as one column with [x,y,z] array (slower)</p> <code>False</code> <code>materialization_version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>timestamp</code> <code>datetime</code> <p>Timestamp to query, by default None. If passsed will do a live query. Error if also passing a materialization version</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>Toggle to return metadata (default True), by default True. If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>merge_reference</code> <code>bool</code> <p>Toggle to automatically join reference table, by default True. If True, metadata will be queries and if its a reference table it will perform a join on the reference table to return the rows of that</p> <code>True</code> <code>desired_resolution</code> <code>Iterable[float]</code> <p>Desired resolution you want all spatial points returned in, by default None. If None, defaults to one specified in client, if that is None then points are returned as stored in the table and should be in the resolution specified in the table metadata</p> <code>None</code> <code>get_counts</code> <code>bool</code> <p>Whether to get counts of the query, by default False</p> <code>False</code> <code>random_sample</code> <code>int</code> <p>If given, will do a tablesample of the of the table to return that many annotations</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings, by default True</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.query_view","title":"<code>query_view(view_name, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, return_df=True, split_positions=False, materialization_version=None, metadata=True, merge_reference=True, desired_resolution=None, get_counts=False, random_sample=None)</code>","text":"<p>Generic query on a view</p> <p>Parameters:</p> Name Type Description Default <code>view_name</code> <code>str</code> <p>View to query</p> required <code>filter_in_dict</code> <code>dict</code> <p>Keys are column names, values are allowed entries, by default None</p> <code>None</code> <code>filter_out_dict</code> <code>dict</code> <p>Keys are column names, values are not allowed entries, by default None</p> <code>None</code> <code>filter_equal_dict</code> <code>dict</code> <p>Keys are column names, values are specified entry, by default None</p> <code>None</code> <code>filter_greater_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive lower-bound, by default None</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive lower-bound, by default None</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict</code> <p>Keys are column names, values are bounding boxes expressed in units of the voxel_resolution of this dataset. Bounding box is [[min_x, min_y,min_z],[max_x, max_y, max_z]], by default None</p> <code>None</code> <code>filter_regex_dict</code> <code>dict</code> <p>Keys are column names, values are regex strings, by default None</p> <code>None</code> <code>select_columns</code> <code>list of str</code> <p>Columns to select, by default None</p> <code>None</code> <code>offset</code> <code>int</code> <p>Result offset to use, by default None. Will only return top K results.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Maximum results to return (server will set upper limit, see get_server_config), by default None</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query, by default None. If None, defaults to one specified in client.</p> <code>None</code> <code>return_df</code> <code>bool</code> <p>Whether to return as a dataframe, by default True. If False, data is returned as json (slower).</p> <code>True</code> <code>split_positions</code> <code>bool</code> <p>Whether to break position columns into x,y,z columns, by default False. If False data is returned as one column with [x,y,z] array (slower)</p> <code>False</code> <code>materialization_version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>Toggle to return metadata (default True), by default True. If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>merge_reference</code> <code>bool</code> <p>Toggle to automatically join reference table, by default True. If True, metadata will be queries and if its a reference table it will perform a join on the reference table to return the rows of that</p> <code>True</code> <code>desired_resolution</code> <code>Iterable[float]</code> <p>Desired resolution you want all spatial points returned in, by default None. If None, defaults to one specified in client, if that is None then points are returned as stored in the table and should be in the resolution specified in the table metadata</p> <code>None</code> <code>get_counts</code> <code>bool</code> <p>Whether to get counts of the query, by default False</p> <code>False</code> <code>random_sample</code> <code>int</code> <p>If given, will do a tablesample of the of the table to return that many annotations</p> <code>None</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.synapse_query","title":"<code>synapse_query(pre_ids=None, post_ids=None, bounding_box=None, bounding_box_column='post_pt_position', timestamp=None, remove_autapses=True, include_zeros=True, limit=None, offset=None, split_positions=False, desired_resolution=None, materialization_version=None, synapse_table=None, datastack_name=None, metadata=True)</code>","text":"<p>Convenience method for querying synapses.</p> <p>Will use the synapse table specified in the info service by default. It will also remove autapses by default. NOTE: This is not designed to allow querying of the entire synapse table. A query with no filters will return only a limited number of rows (configured by the server) and will do so in a non-deterministic fashion. Please contact your dataset administrator if you want access to the entire table.</p> <p>Parameters:</p> Name Type Description Default <code>pre_ids</code> <code>Union[int, Iterable, ndarray]</code> <p>Pre-synaptic cell(s) to query.</p> <code>None</code> <code>post_ids</code> <code>Union[int, Iterable, ndarray]</code> <p>Post-synaptic cell(s) to query.</p> <code>None</code> <code>bounding_box</code> <code>Optional[Union[list, ndarray]]</code> <p>[[min_x, min_y, min_z],[max_x, max_y, max_z]] bounding box to filter synapse locations. Expressed in units of the voxel_resolution of this dataset.</p> <code>None</code> <code>bounding_box_column</code> <code>str</code> <p>Which synapse location column to filter by.</p> <code>'post_pt_position'</code> <code>timestamp</code> <code>datetime</code> <p>Timestamp to query. If passed recalculate query at timestamp, do not pass with materialization_version.</p> <code>None</code> <code>remove_autapses</code> <code>bool</code> <p>Whether to remove autapses from query results.</p> <code>True</code> <code>include_zeros</code> <code>bool</code> <p>Whether to include synapses to/from id=0 (out of segmentation).</p> <code>True</code> <code>limit</code> <code>int</code> <p>Number of synapses to limit. Server-side limit still applies.</p> <code>None</code> <code>offset</code> <code>int</code> <p>Number of synapses to offset query.</p> <code>None</code> <code>split_positions</code> <code>bool</code> <p>Whether to split positions into separate columns, True is faster.</p> <code>False</code> <code>desired_resolution</code> <code>Iterable[float]</code> <p>List or array of the desired resolution you want queries returned in useful for materialization queries.</p> <code>None</code> <code>materialization_version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>Whether to attach metadata to dataframe in the df.attr dictionary.</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>Results of query.</p>"},{"location":"api/schema/","title":"client.schema","text":"<p>Client for interacting with the schema service.</p> <p>Methods:</p> Name Description <code>get_schemas</code> <p>Get the available schema types</p> <code>schema_definition</code> <p>Get the definition of a specified schema_type</p> <code>schema_definition_all</code> <p>Get the definition of all schema_types</p> <code>schema_definition_multi</code> <p>Get the definition of multiple schema_types</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.get_schemas","title":"<code>get_schemas()</code>","text":"<p>Get the available schema types</p> <p>Returns:</p> Type Description <code>list</code> <p>List of schema types available on the Schema service.</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.schema_definition","title":"<code>schema_definition(schema_type)</code>","text":"<p>Get the definition of a specified schema_type</p> <p>Parameters:</p> Name Type Description Default <code>schema_type</code> <code>str</code> <p>Name of a schema_type</p> required <p>Returns:</p> Type Description <code>json</code> <p>Schema definition</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.schema_definition_all","title":"<code>schema_definition_all()</code>","text":"<p>Get the definition of all schema_types</p> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of schema definitions. Keys are schema names, values are definitions.</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.schema_definition_multi","title":"<code>schema_definition_multi(schema_types)</code>","text":"<p>Get the definition of multiple schema_types</p> <p>Parameters:</p> Name Type Description Default <code>schema_types</code> <code>list</code> <p>List of schema names</p> required <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of schema definitions. Keys are schema names, values are definitions.</p>"},{"location":"api/skeleton/","title":"client.skeleton","text":"<p>Client for interacting with the skeleton service.</p> <p>Methods:</p> Name Description <code>get_skeleton</code> <p>Gets basic skeleton information for a datastack</p> <code>get_cache_contents</code> <p>Mirror CloudFiles.list() for skeletons as a pass-through interface to the underlying service and bucket.</p> <code>skeletons_exist</code> <p>Confirm or deny that a set of root ids have H5 skeletons in the cache.</p> <code>get_bulk_skeletons</code> <p>Generates skeletons for a list of root ids without retrieving them.</p> <code>generate_bulk_skeletons_async</code> <p>Generates skeletons for a list of root ids without retrieving them.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.get_skeleton","title":"<code>get_skeleton(root_id, datastack_name=None, skeleton_version=3, output_format='dict', log_warning=True, verbose_level=0)</code>","text":"<p>Gets basic skeleton information for a datastack</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>The root id of the skeleton to retrieve</p> required <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check</p> <code>None</code> <code>skeleton_version</code> <code>int</code> <p>The skeleton version to generate and retrieve. Options are documented in SkeletonService. Use 0 for Neuroglancer-compatibility. Use -1 for latest.</p> <code>3</code> <code>output_format</code> <code>string</code> <p>The format to retrieve. Options are: - 'dict': A dictionary - 'swc': A pandas DataFrame</p> <code>'dict'</code> <p>Returns:</p> Type Description <p>Skeleton of the requested type. See <code>output_format</code> for details.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.get_cache_contents","title":"<code>get_cache_contents(datastack_name=None, skeleton_version=3, root_id_prefixes=0, limit=0, log_warning=True)</code>","text":"<p>Mirror CloudFiles.list() for skeletons as a pass-through interface to the underlying service and bucket.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.skeletons_exist","title":"<code>skeletons_exist(datastack_name=None, skeleton_version=3, root_ids=0, log_warning=True)</code>","text":"<p>Confirm or deny that a set of root ids have H5 skeletons in the cache.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.get_bulk_skeletons","title":"<code>get_bulk_skeletons(root_ids, datastack_name=None, skeleton_version=3, output_format='dict', generate_missing_skeletons=False, log_warning=True, verbose_level=0)</code>","text":"<p>Generates skeletons for a list of root ids without retrieving them.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>List</code> <p>A list of root ids of the skeletons to generate</p> required <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check</p> <code>None</code> <code>skeleton_version</code> <code>int</code> <p>The skeleton version to generate. Use 0 for Neuroglancer-compatibility. Use -1 for latest.</p> <code>3</code>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.generate_bulk_skeletons_async","title":"<code>generate_bulk_skeletons_async(root_ids, datastack_name=None, skeleton_version=None, log_warning=True, verbose_level=0)</code>","text":"<p>Generates skeletons for a list of root ids without retrieving them.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>List</code> <p>A list of root ids of the skeletons to generate</p> required <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check</p> <code>None</code> <code>skeleton_version</code> <code>int</code> <p>The skeleton version to generate. Use 0 for Neuroglancer-compatibility. Use -1 for latest.</p> <code>None</code> <p>Returns:</p> Type Description <code>float</code> <p>The estimated time in seconds to generate all skeletons (a comparable message will be output to the console prior to return).</p>"},{"location":"api/state/","title":"client.state","text":"<p>Client to interface with the JSON state service.</p> <p>Methods:</p> Name Description <code>build_neuroglancer_url</code> <p>Build a URL for a Neuroglancer deployment that will automatically retrieve specified state.</p> <code>get_neuroglancer_info</code> <p>Get the info field from a Neuroglancer deployment</p> <code>get_property_json</code> <p>Download a Neuroglancer JSON state</p> <code>get_state_json</code> <p>Download a Neuroglancer JSON state</p> <code>save_state_json_local</code> <p>Save a Neuroglancer JSON state to a JSON file locally.</p> <code>upload_property_json</code> <p>Upload a Neuroglancer JSON state</p> <code>upload_state_json</code> <p>Upload a Neuroglancer JSON state</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p> <code>state_service_endpoint</code> <code>str</code> <p>Endpoint URL for posting JSON state</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.state_service_endpoint","title":"<code>state_service_endpoint: str</code>  <code>property</code>","text":"<p>Endpoint URL for posting JSON state</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.build_neuroglancer_url","title":"<code>build_neuroglancer_url(state_id, ngl_url=None, target_site=None, static_url=False, format_properties=False)</code>","text":"<p>Build a URL for a Neuroglancer deployment that will automatically retrieve specified state. If the datastack is specified, this is prepopulated from the info file field \"viewer_site\". If no ngl_url is specified in either the function or the client, a fallback neuroglancer deployment is used.</p> <p>Parameters:</p> Name Type Description Default <code>state_id</code> <code>int</code> <p>State id to retrieve</p> required <code>ngl_url</code> <code>str</code> <p>Base url of a neuroglancer deployment. If None, defaults to the value for the datastack or the client. As a fallback, a default deployment is used.</p> <code>None</code> <code>target_site</code> <code>seunglab or cave - explorer or mainline or None</code> <p>Set this to 'seunglab' for a seunglab deployment, or either 'cave-explorer'/'mainline' for a google main branch deployment. If None, checks the info field of the neuroglancer endpoint to determine which to use. Default is None.</p> <code>None</code> <code>static_url</code> <code>bool</code> <p>If True, treats \"state_id\" as a static URL directly to the JSON and does not use the state service.</p> <code>False</code> <code>format_properties</code> <code>bool</code> <p>If True, formats the url as a segment_properties info file</p> <code>False</code> <p>Returns:</p> Type Description <code>str</code> <p>The full URL requested</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.get_neuroglancer_info","title":"<code>get_neuroglancer_info(ngl_url=None)</code>","text":"<p>Get the info field from a Neuroglancer deployment</p> <p>Parameters:</p> Name Type Description Default <code>ngl_url</code> <code>str(optional)</code> <p>URL to a Neuroglancer deployment. If None, defaults to the value for the datastack or the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>JSON-formatted info field from the Neuroglancer deployment</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.get_property_json","title":"<code>get_property_json(state_id)</code>","text":"<p>Download a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>state_id</code> <code>int</code> <p>ID of a JSON state uploaded to the state service.</p> required <p>Returns:</p> Type Description <code>dict</code> <p>JSON specifying a Neuroglancer state.</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.get_state_json","title":"<code>get_state_json(state_id)</code>","text":"<p>Download a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>state_id</code> <code>int or string</code> <p>ID of a JSON state uploaded to the state service. If a string, treats this as the url to the json state.</p> required <p>Returns:</p> Type Description <code>dict</code> <p>JSON specifying a Neuroglancer state.</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.save_state_json_local","title":"<code>save_state_json_local(json_state, filename, overwrite=False)</code>","text":"<p>Save a Neuroglancer JSON state to a JSON file locally.</p> <p>Parameters:</p> Name Type Description Default <code>json_state</code> <code>dict</code> <p>Dict representation of a neuroglancer state</p> required <code>filename</code> <code>str</code> <p>Filename to save the state to</p> required <code>overwrite</code> <code>bool</code> <p>Whether to overwrite the file if it exists. Default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>None</code>"},{"location":"api/state/#caveclient.jsonservice.JSONService.upload_property_json","title":"<code>upload_property_json(property_json, state_id=None, max_size=2500000)</code>","text":"<p>Upload a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>propery_json</code> <code>dict</code> <p>Dict representation of a neuroglancer segment properties json</p> required <code>state_id</code> <code>int</code> <p>ID of a JSON state uploaded to the state service. Using a state_id is an admin feature.</p> <code>None</code> <code>max_size</code> <code>Optional[int]</code> <p>Maximum size in bytes for the data to upload. Default is 2.5MB. Set to None for no limit.</p> <code>2500000</code> <p>Returns:</p> Type Description <code>int</code> <p>state_id of the uploaded JSON state</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.upload_state_json","title":"<code>upload_state_json(json_state, state_id=None, timestamp=None)</code>","text":"<p>Upload a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>json_state</code> <code>dict</code> <p>Dict representation of a neuroglancer state</p> required <code>state_id</code> <code>int</code> <p>ID of a JSON state uploaded to the state service. Using a state_id is an admin feature.</p> <code>None</code> <code>timestamp</code> <code>Optional[time]</code> <p>Timestamp for json state date. Requires state_id.</p> <code>None</code> <p>Returns:</p> Type Description <code>int</code> <p>state_id of the uploaded JSON state</p>"},{"location":"api/testing/","title":"caveclient.tools.testing","text":"<p>Functions:</p> Name Description <code>CAVEclientMock</code> <p>Created a mocked CAVEclient function for testing using the responses library to mock</p> <code>get_materialiation_info</code> <p>Get the materialization versions and version metadata for the materialization service.</p> <code>get_server_versions</code> <p>Get the server versions for the services used in testing.</p> <code>get_server_information</code> <p>Generate the datastack name and server locations used in testing.</p> <code>get_api_versions</code> <p>Get the API versions for the services used in testing.</p> <code>default_info</code> <p>Generate a info service info file for testing</p>"},{"location":"api/testing/#caveclient.tools.testing.CAVEclientMock","title":"<code>CAVEclientMock(datastack_name=None, global_server=None, local_server=None, info_file=None, chunkedgraph=False, chunkedgraph_server_version=DEFAULT_CHUNKEDGRAPH_SERVER_VERSION, chunkedgraph_api_versions=None, materialization=False, materialization_server_version=DEFAULT_MATERIALIZATION_SERVER_VERSON, materialization_api_versions=None, available_materialization_versions=None, set_version=None, set_version_metadata=None, json_service=False, json_service_server_version=DEFAULT_JSON_SERVICE_SERVER_VERSION, skeleton_service=False, skeleton_service_server_version=DEFAULT_SKELETON_SERVICE_SERVER_VERSION, schema_api_versions=None, l2cache=False, l2cache_disabled=False, global_only=False)</code>","text":"<p>Created a mocked CAVEclient function for testing using the responses library to mock the server responses. This function returns a drop-in replacement for the <code>CAVEclient</code> function that will be able to initialize itself and selected individual service clients with the selected options.</p> <p>Note that the test configuration is intended to be purely for pytest purposes and should not actually result in calls to active endpoints.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>Name of the test datastack, by default None</p> <code>None</code> <code>global_server</code> <code>str</code> <p>Test global server address, by default None</p> <code>None</code> <code>local_server</code> <code>str</code> <p>Test local server address, by default None</p> <code>None</code> <code>info_file</code> <code>dictionary</code> <p>Info service dictionary for the datastack, by default None</p> <code>None</code> <code>chunkedgraph</code> <code>bool</code> <p>If True, configures the client to initialize a chunkedgraph subclient, by default False</p> <code>False</code> <code>chunkedgraph_server_version</code> <code>str</code> <p>Sets the value of the chunkedgraph server version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_CHUNKEDGRAPH_SERVER_VERSION.</p> <code>DEFAULT_CHUNKEDGRAPH_SERVER_VERSION</code> <code>chunkedgraph_api_versions</code> <code>list</code> <p>List of chunkedgraph API versions that the chunkedgraph client thinks exists, by default None. If None, returns the value in CHUNKEDGRAPH_API_VERSIONS.</p> <code>None</code> <code>materialization</code> <code>bool</code> <p>If True, configures the client to initalize a materialization subclient, by default False Note that materialization being set to True will also configure the chunkedgraph client.</p> <code>False</code> <code>materialization_server_version</code> <code>str</code> <p>Sets the value of the materialization server version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_MATERIALIZATION_SERVER_VERSON.</p> <code>DEFAULT_MATERIALIZATION_SERVER_VERSON</code> <code>available_materialization_versions</code> <code>list</code> <p>List of materialization database versions that the materialization client thinks exists, by default None. If None, returns the value in DEFAULT_MATERIALIZATION_VERSONS.</p> <code>None</code> <code>materialization_api_versions</code> <code>list</code> <p>List of materialization API versions that the materialization client thinks exists, by default None. If None, returns the value in MATERIALIZATION_API_VERSIONS.</p> <code>None</code> <code>set_version</code> <code>Optional[int]</code> <p>If set, will set the version of the materialization server to the value of set_version, by default None. To work, this version must be in the list of available materialization versions.</p> <code>None</code> <code>set_version_metadata</code> <code>Optional[dict]</code> <p>If set, will set the version metadata of the materialization server to the value of set_version_metadata. Default value is in DEFAULT_MATERIALIZATION_VERSION_METADATA.</p> <code>None</code> <code>json_service</code> <code>bool</code> <p>If True, configures the client to initalize a materialization subclient, by default False</p> <code>False</code> <code>json_service_server_version</code> <code>_type_</code> <p>Sets the value of the json state server version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_JSON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_JSON_SERVICE_SERVER_VERSION</code> <code>skeleton_service</code> <code>bool</code> <p>If True, configures the client to initalize a skeleton service subclient, by default False</p> <code>False</code> <code>skeleton_service_server_version</code> <code>_type_</code> <p>Sets the value of the skeleton service version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_SKELETON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_SKELETON_SERVICE_SERVER_VERSION</code> <code>l2cache</code> <code>bool</code> <p>If True, configures the client to initialize an l2cache subclient, by default False Note that l2cache being set to True will also configure the chunkedgraph client.</p> <code>False</code> <code>l2cache_disabled</code> <code>bool</code> <p>If True, allows a subclient to be initialized, but emulates a situation without an L2 cache, by default False Only used if l2cache is True.</p> <code>False</code> <code>global_only</code> <code>bool</code> <p>If True, only initializes the global services and does not use a datastack, by default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>CAVEclient</code> <p>A mocked and initialized CAVEclient object for testing</p> <p>Examples:</p> <p>To make a basic pytest fixture to test chunkedgraph features with an initialized CAVEclient object in your pytest conftest.py file:</p> <pre><code>import pytest\nfrom caveclient.tools.testing import CAVEclientMock\n\ntest_datastack = \"test_stack\"\ntest_global_server = \"https://test.cave.com\"\ntest_local_server = \"https://local.cave.com\"\n\n@pytest.fixture()\ndef test_client():\n    return CAVEclientMock(\n        datastack_name=test_datastack,\n        global_server=test_global_server,\n        local_server=test_local_server,\n        chunkedgraph=True,\n    )\n\nYou can also create more complex fixtures with multiple services initialized and specific server versions:\n\n```python\n@pytest.fixture()\ndef fancier_test_client():\n    return CAVEclientMock(\n        datastack_name=test_datastack,\n        global_server=test_global_server,\n        local_server=test_local_server,\n        chunkedgraph=True,\n        chunkedgraph_server_version=\"3.0.2\",\n        materialization=True,\n        materialization_server_version=\"4.21.4\",\n        l2cache=True,\n    )\n</code></pre>"},{"location":"api/testing/#caveclient.tools.testing.get_materialiation_info","title":"<code>get_materialiation_info(materialization_versions=DEFAULT_MATERIALIZATION_VERSONS, version_metadata=DEFAULT_MATERIALIZATION_VERSION_METADATA)</code>","text":"<p>Get the materialization versions and version metadata for the materialization service.</p> <p>Parameters:</p> Name Type Description Default <code>materialization_versions</code> <code>list</code> <p>List of materialization database versions that the materialization client thinks exists, by default DEFAULT_MATERIALIZATION_VERSONS.</p> <code>DEFAULT_MATERIALIZATION_VERSONS</code> <code>version_metadata</code> <code>dict</code> <p>Version metadata for the materialization service, by default DEFAULT_MATERIALIZATION_VERSION_METADATA.</p> <code>DEFAULT_MATERIALIZATION_VERSION_METADATA</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"materialization_versions\", \"version_metadata\".</p>"},{"location":"api/testing/#caveclient.tools.testing.get_server_versions","title":"<code>get_server_versions(chunkedgraph_version=DEFAULT_CHUNKEDGRAPH_SERVER_VERSION, materialization_version=DEFAULT_MATERIALIZATION_SERVER_VERSON, skeleton_service_version=DEFAULT_SKELETON_SERVICE_SERVER_VERSION, json_service_version=DEFAULT_JSON_SERVICE_SERVER_VERSION)</code>","text":"<p>Get the server versions for the services used in testing.</p> <p>Parameters:</p> Name Type Description Default <code>chunkedgraph_version</code> <code>str</code> <p>Version of the chunkedgraph server, by default DEFAULT_CHUNKEDGRAPH_SERVER_VERSION.</p> <code>DEFAULT_CHUNKEDGRAPH_SERVER_VERSION</code> <code>materialization_version</code> <code>str</code> <p>Version of the materialization server, by default DEFAULT_MATERIALIZATION_SERVER_VERSON.</p> <code>DEFAULT_MATERIALIZATION_SERVER_VERSON</code> <code>skeleton_service_version</code> <code>str</code> <p>Version of the skeleton service server, by default DEFAULT_SKELETON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_SKELETON_SERVICE_SERVER_VERSION</code> <code>json_service_version</code> <code>str</code> <p>Version of the json service server, by default DEFAULT_JSON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_JSON_SERVICE_SERVER_VERSION</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"chunkedgraph_version\", \"materialization_version\", \"skeleton_service_version\", \"json_service_version\". Values are Version objects from packaging.versions.</p>"},{"location":"api/testing/#caveclient.tools.testing.get_server_information","title":"<code>get_server_information(datastack_name=TEST_DATASTACK, global_server=TEST_GLOBAL_SERVER, local_server=TEST_LOCAL_SERVER)</code>","text":"<p>Generate the datastack name and server locations used in testing.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>Datastack value, by default the value in TEST_DATASTACK.</p> <code>TEST_DATASTACK</code> <code>global_server</code> <code>str</code> <p>Server for global services, by default TEST_GLOBAL_SERVER.</p> <code>TEST_GLOBAL_SERVER</code> <code>local_server</code> <code>str</code> <p>Server for local services, by default TEST_LOCAL_SERVER.</p> <code>TEST_LOCAL_SERVER</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"datastack_name\", \"local_server\", \"global_server\".</p>"},{"location":"api/testing/#caveclient.tools.testing.get_api_versions","title":"<code>get_api_versions(chunkedgraph_api_versions=CHUNKEDGRAPH_API_VERSIONS, materialization_api_versions=MATERIALIZATION_API_VERSIONS, schema_api_versions=SCHEMA_API_VERSIONS)</code>","text":"<p>Get the API versions for the services used in testing.</p> <p>Parameters:</p> Name Type Description Default <code>chunkedgraph_api_versions</code> <code>list</code> <p>List of chunkedgraph API versions that the chunkedgraph client thinks exists, by default CHUNKEDGRAPH_API_VERSIONS.</p> <code>CHUNKEDGRAPH_API_VERSIONS</code> <code>materialization_api_versions</code> <code>list</code> <p>List of materialization API versions that the materialization client thinks exists, by default MATERIALIZATION_API_VERSIONS.</p> <code>MATERIALIZATION_API_VERSIONS</code> <code>schema_api_versions</code> <code>list</code> <p>List of schema API versions that the schema client thinks exists, by default SCHEMA_API_VERSIONS.</p> <code>SCHEMA_API_VERSIONS</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"chunkedgraph_api_versions\", \"materialization_api_versions\", \"schema_api_versions\".</p>"},{"location":"api/testing/#caveclient.tools.testing.default_info","title":"<code>default_info(local_server=TEST_LOCAL_SERVER)</code>","text":"<p>Generate a info service info file for testing</p> <p>Parameters:</p> Name Type Description Default <code>local_server</code> <code>str</code> <p>Name of the local service, by default the value in TEST_LOCAL_SERVER.</p> <code>TEST_LOCAL_SERVER</code> <p>Returns:</p> Type Description <code>dict</code> <p>Info file for the datastack.</p>"},{"location":"tutorials/","title":"Getting Started","text":"<p>AnnotationFramework client is a package for simplifying interactions with HTML services associated with the CAVE (Connectome Annotation Versioning Engine).</p> <p>For a larger introduction to CAVE and its services, see the main GitHub organization site: https://github.com/CAVEconnectome</p>"},{"location":"tutorials/#installation","title":"Installation","text":"<p>The CAVEclient can be installed with pip:</p> <pre><code>$ pip install caveclient\n</code></pre>"},{"location":"tutorials/#assumptions","title":"Assumptions","text":"<p>The code is setup to work flexibly with any deployment of these services, but you need to specify the server_address if that address is not https://globalv1.daf-apis.com/ for each client when initializing it. Similarly, the clients can query the info service for metadata to simplify the interaction with a datastack, but you have to specify a datastack name.</p>"},{"location":"tutorials/advanced/","title":"Advanced Usage","text":""},{"location":"tutorials/advanced/#changing-session-configurations","title":"Changing session configurations","text":"<p>It is possible to change details of how a client talks to the various servers it needs to interface with. For instance, the <code>CAVEclient</code> will attempt to retry specific failed requests to the server, but will only try a specific number of times, and will only wait specific amounts of time between retries. These values can be changed via the <code>set_session_defaults</code> method. For instance, to change the number of retries to 5, and to increase the delay between subsequent retries, you could do:</p> <pre><code>from caveclient import set_session_defaults\n\nset_session_defaults(max_retries=5, backoff_factor=0.5)\n</code></pre> <p>Note that this needs to happen before initializing the client for this to work properly. Some of these parameters are also adjustable at the client level.</p> <p>To view the current session defaults, you can use the <code>get_session_defaults</code> method:</p> <pre><code>from caveclient import get_session_defaults\n\nclient.get_session_defaults()\n</code></pre> <p>More information on the available parameters can be found in the API documentation.</p>"},{"location":"tutorials/advanced/#writing-unit-tests-using-caveclient","title":"Writing unit tests using caveclient","text":"<p>If you have a module that relies on caveclient and you want to write unit tests for it, you can use the <code>caveclient.testing</code> module to help you create an initialized, mocked client with user-specified options. This helps deal with the complex conversation that caveclient has with various services when initializing itself so that you get the appropriate functionality. The function <code>CAVEclientMock</code> offers a number of options for how to build a client, including server versions for various services, available API versions, and more. Importantly, the module provides sensible defaults but also allows you to override them as needed. Note that using testing requires the responses to be installed, which is not a dependency of caveclient.</p> <p>The <code>CAVEclientMock</code> function requires you to specify which services you expect to call on in order to ensure that your code is working as expected. For example, to create a pytest fixture that would utilize the <code>chunkedgraph</code> module, but otherwise use default nonsense test values, you could include this in your <code>conftest.py</code>:</p> <pre><code>import pytest\nfrom caveclient.tools.testing import CAVEclientMock\n\n@pytest.fixture()\ndef test_client():\n    return CAVEclientMock(\n        chunkedgraph=True,\n    )\n</code></pre> <p>Then, in your test module, you can use the <code>test_client</code> fixture to get a client. Note that after mocked initialization, the client will attempt normal network requests and thus you should mock the responses. If you only care to get a specific value back from a given function, you can use pytest-mock to mock the response to a given function call. For example, if you have a function <code>do_something_with_roots</code> that takes a caveclient that uses the <code>get_roots</code> function, you could mock the <code>get_roots</code> function so that it return a specific value:</p> <pre><code>from pytest_mock import mocker\nfrom conftest import test_client\nfrom my_module import do_something_with_roots\n\ndef test_get_roots(mocker, test_client):\n    mocker.patch.object(test_client.chunkedgraph, 'get_roots', return_value=[1, 2, 3])\n    test_data = # Something appropriate\n    test_output = do_something_with_roots(test_client, test_data)\n    assert test_output == # The expected answer\n</code></pre> <p>Note that if you your own datastack <code>info_file</code> that has a different <code>local_server</code> address than the default value (<code>TEST_LOCAL_SERVER</code> variable, which defaults to <code>https://local.cave.com</code>) you will also need to specify a <code>local_server</code> with the same value as in the <code>local_server</code> field of your info file.</p>"},{"location":"tutorials/advanced/#specifying-conditions","title":"Specifying conditions","text":"<p>While sensible defaults are provided, you can also specify things like server versions to make sure your code works with the versions of the services you expect. For example, let's make a richer mock client that specifies the server versions for the <code>chunkedgraph</code>, <code>materailization</code>, and <code>l2cache</code> services:</p> <pre><code>@pytest.fixture()\ndef version_specified_client():\n    return CAVEclientMock(\n        chunkedgraph=True,\n        chunkedgraph_server_version='3.0.1',\n        materialization=True,\n        materialization_server_version='2.0.0',\n        l2cache=True,\n    )\n</code></pre> <p>Note that some services like <code>l2cache</code> do not currently use a server version to offer different functionality, and this value is not exposed for them currently. See the API documentation for more information.</p> <p>You can also override default values like <code>server_address</code> or <code>datastack_name</code>: <pre><code>@pytest.fixture()\ndef server_specified_client():\n    return CAVEclientMock(\n        datastack_name='my_datastack',\n        server_address='http://my.server.com',\n        chunkedgraph=True,\n        materialization=True,\n    )\n</code></pre></p> <p>If you want to get access to the various default values of server version, datastack name, datastack info file, and api versions, you can use functions <code>get_server_versions</code>, <code>get_server_information</code>, <code>default_info</code>, <code>get_api_versions</code> respectively. Each of these functions will return a dictionary of the values that can be used as a kwargs input into CAVEclientMock. If you specify your own override values, it will take precedence over the default value and you can just use the dictionary in your tests. See the caveclient tests for an example of how to use these functions.</p>"},{"location":"tutorials/annotation/","title":"Annotation","text":"<p>The AnnotationClient is used to interact with the AnnotationEngine service to create tables from existing schema, upload new data, and download existing annotations. Note that annotations in the AnnotationEngine are not linked to any particular segmentation, and thus do not include any root ids. An annotation client is accessed with <code>client.annotation</code>.</p>"},{"location":"tutorials/annotation/#getting-existing-tables","title":"Getting existing tables","text":"<p>A list of the existing tables for the datastack can be found with get_tables().</p> <pre><code>all_tables = client.annotation.get_tables()\nall_tables[0]\n</code></pre> <p>Each table has three main properties that can be useful to know:</p> <ul> <li><code>table_name</code> : The table name, used to refer to it when uploading or   downloading annotations. This is also passed through to the table in   the Materialized database.</li> <li><code>schema_name</code> : The name of the table's schema from   EMAnnotationSchemas (see below).</li> <li><code>max_annotation_id</code> : An upper limit on the number of annotations   already contained in the table.</li> </ul>"},{"location":"tutorials/annotation/#downloading-annotations","title":"Downloading annotations","text":"<p>You can download the JSON representation of a data point through the get_annotation() method. This can be useful if you need to look up information on unmaterialized data, or to see what a properly templated annotation looks like.</p> <pre><code>table_name = all_tables[0]['table_name']      # 'ais_analysis_soma'\nannotation_id = 100\nclient.annotation.get_annotation(annotation_ids=annotation_id, table_name=table_name)\n</code></pre>"},{"location":"tutorials/annotation/#create-a-new-table","title":"Create a new table","text":"<p>One can create a new table with a specified schema with the create_table() method:</p> <pre><code>client.annotation.create_table(table_name='test_table',\n                               schema_name='microns_func_coreg',\n                               voxel_resolution = [1,1,1],\n                               description=\"some text to describe your table\")\n</code></pre> <p>The voxel resolution is the units your position columns will be uploaded in [1,1,1] would imply a nm location, where as [4,4,40] would correspond to voxels of that size. If you are uploading points from a neuroglancer session, you want this to match the units of that neuroglancer view.</p> <p>Note there are some optional metadata parameters to create_table()</p> <ul> <li><code>notice_text</code> : This is text that will show up to users who access   this data as a warning. This could be used to warn users that the   data is not complete or checked yet, or to advertise that a   particular publication should be cited when using this table.</li> <li><code>read_permission</code> : one of \"PRIVATE\" which means only you can read   data in this table. \"PUBLIC\" (default) which means anyone can read   this table that has read permissions to this dataset. So if and only   if you can read the segmentation results of this data, you can read   this table. \"GROUP\" which means that you must share a common group   with this user for them to be able to read. We need to make a way to   discover what groups you are in and who you share groups with.</li> <li><code>write_permission</code>: one of \"PRIVATE\" (default), which means only   you can write to this table. \"PUBLIC\" which means anyone can write   to this table that has write permissions to this dataset. Note   although this means anyone can add data, no annotations are ever   truly overwritten. \"GROUP\" which means that you must share a   common group with this user for them to be able to write. We need to   make a way to discover what groups you are in and who you share   groups with.</li> </ul> <p>If you change your mind about what you want for metadata, some but not all fields can be updated with update_metadata(). This includes the description, the notice_text, and the permissions, but not the name, schema or voxel resolution.</p> <pre><code># to update description\nclient.annotation.update_metadata(table_name='test_table',\n                                  description=\"a new description for my table\")\n\n# to make your table readable by anybody who can read this dataset\nclient.annotation.update_metadata(table_name='test_table',\n                                  notice_text=\"This table isn't done yet, don't trust it. Contact me\")\n\n# to make your table readable by anybody who can read this dataset\nclient.annotation.update_metadata(table_name='test_table',\n                                  read_permisison=\"PUBLIC\")\n</code></pre> <p>New data can be generated as a dict or list of dicts following the schema and uploaded with <code>post_annotation</code>. For example, a <code>microns_func_coreg</code> point needs to have:</p> <ul> <li><code>type</code> set to <code>microns_func_coreg</code></li> <li><code>pt</code> set to a dict with <code>position</code> as a key and   the xyz location as a value.</li> <li><code>func_id</code> set to an integer.</li> </ul> <p>The following could would create a new annotation and then upload it to the service. Note that you get back the annotation id(s) of what you uploaded.</p> <pre><code>new_data = {'type': 'microns_func_coreg',\n            'pt': {'position': [1,2,3]},\n            'func_id': 0}\nclient.annotation.post_annotation(table_name='test_table', data=[new_data])\n</code></pre> <p>There are methods to simplify annotation uploads if you have a pandas dataframe whose structure mirrors the struction of the annotation schema you want to upload</p> <pre><code>import pandas as pd\n\ndf = pd.DataFrame([{'id':0,\n         'type': 'microns_func_coreg',\n         'pt_position': [1,2,3]},\n         'func_id': 0},\n        {'id':1,\n        'type': 'microns_func_coreg',\n        'pt_position': [3,2,1]},\n        'func_id': 2}])\nclient.annotation.post_annotation_df('test_table', df)\n</code></pre> <p>Note that here I specified the IDs of my annotations, which you can do, but then its up to you to assure that the IDs don't collide with other IDs. If you leave them blank then the service will assign the IDs for you.</p> <p>There is a similar method for updating update_annotation_df()</p>"},{"location":"tutorials/annotation/#staged-annotations","title":"Staged Annotations","text":"<p>Staged annotations help ensure that the annotations you post follow the appropriate schema, both by providing guides to the field names and locally validating against a schema before uploading. The most common use case for staged annotations is to create a StagedAnnotation object for a given table, then add annotations to it individually or as a group, and finally upload to the annotation table.</p> <p>To get a StagedAnnotation object, you can start with either a table name or a schema name. Here, we'll assume that there's already a table called \"my_table\" that is running a \"cell_type_local\" schema. If we want to add new annotations to the table, we simply use the table name with stage_annotations().</p> <pre><code>stage = client.annotation.stage_annotations(\"my_table\")\n</code></pre> <p>This <code>stage</code> object retrieves the schema for the table and hosts a local collection of annotations. Every time you add an annotation, it is immediately validated against the schema. To add an annotation, use the <code>add</code> method:</p> <pre><code>stage.add(\n    cell_type = \"pyramidal_cell\",\n    classification_system=\"excitatory\",\n    pt_position=[100,100,10],\n)\n</code></pre> <p>The argument names derive from fields in the schema and you must provide all required fields. Any number of annotations can be added to the stage. A dataframe of annotations can also be added with <code>stage.add_dataframe</code>, and requires an exact match between column names and schema fields. The key difference between this and posting a dataframe directly is that annotations added to a StagedAnnotations are validated locally, allowing any issues to be caught before uploading.</p> <p>You can see the annotations as a list of dictionary records with <code>stage.annotation_list</code> or as a Pandas dataframe with <code>stage.annotation_dataframe</code>. Finally, if you initialized the stage with a table name, this information is stored in the <code>stage</code> and you can simply upload it from the client.</p> <pre><code>client.annotation.upload_staged_annotations(stage)\n</code></pre> <p>Updating annotations requires knowing the annotation id of the annotation you are updating, which is not required in the schema otherwise. In order to stage updated annotations, set the <code>update</code> parameter to <code>True</code> when creating the stage.</p> <pre><code>update_stage = client.annotation.stage_annotations(\"my_table\", update=True)\nupdate_stage.add(\n    id=1,\n    cell_type = \"stellate_cell\",\n    classification_system=\"excitatory\",\n    pt_position=[100,100,10],\n)\n</code></pre> <p>The <code>update</code> also informs the framework client to treat the annotations as an update and it will use the appropriate methods automatically when uploading <code>client.annotation.upload_staged_annotations</code>.</p> <p>If you want to specify ids when posting new annotations, <code>id_field</code> can be set to True when creating the StagedAnnotation object. This will enforce an <code>id</code> column but still post the data as new annotations.</p> <p>If you might be adding spatial data in coordinates that might be different than the resolution for the table, you can also set the <code>annotation_resolution</code> when creating the stage. The stage will convert between the resolution you specify for your own annotations and the resolution that the table expects.</p> <pre><code>stage = client.annotation.stage_annotations(\"my_table\", annotation_resolution=[8,8,40])\nstage.add(\n    cell_type='pyramidal_cell',\n    classification_system=\"excitatory\",\n    pt_position=[50,50,10],\n)\n</code></pre>"},{"location":"tutorials/authentication/","title":"Authentication","text":"<p>Authentication tokens are generally needed for programmatic access to our services. The AuthClient handles storing and loading your token or tokens and inserting it into requests in other clients.</p> <p>We can access the auth client from <code>client.auth</code>. Once you have saved a token, you probably won't interact with this client very often, however it has some convenient features for saving new tokens the first time. Let's see if you have a token already. Probably not.</p> <pre><code>client = CAVEclient()\nauth = client.auth\nprint(f\"My current token is: {auth.token}\")\n</code></pre>"},{"location":"tutorials/authentication/#getting-a-new-token","title":"Getting a new token","text":"<p>To get a new token, you will need to manually acquire it. For convenience, the function client.auth.get_new_token() provides instructions for how to get and save the token.</p> <p>By default, the token is saved to <code>~/.cloudvolume/secrets/cave-secret.json</code> as a string under the key <code>token</code>. This makes it compatible by default with Cloudvolume projects, which can come in handy. The following steps will save a token to the default location.</p> <pre><code>auth.get_new_token()\n</code></pre> <pre><code>new_token = 'abcdef1234567890' #This is the text you see after you visit the website.\nauth.save_token(token=new_token)\nprint(f\"My token is now: {auth.token}\")\n</code></pre> <p>Note that requesting a new token will invalidate your previous token on the same project. If you want to use the same token across different computers, you will need to share the same token information.</p>"},{"location":"tutorials/authentication/#loading-saved-tokens","title":"Loading saved tokens","text":"<p>Try opening <code>~/.cloudvolume/secrets/cave-secret.json</code> to see what we just created.</p> <p>If we had wanted to use a different file or a different json key, we could have specified that in auth.save_token.</p> <p>Because we used the default values, this token is used automatically when we initialize a new CAVEclient. If we wanted to use a different token file, token key, or even directly specify a token we could do so here.</p> <pre><code>client = CAVEclient(datastack_name)\nprint(f\"Now my basic token is: {client.auth.token}\")\n\nclient_direct = CAVEclient(datastack_name, auth_token='another_fake_token_678')\nprint(f\"A directly specified token is: {client_direct.auth.token}\")\n</code></pre> <p>If you use a CAVEclient, the AuthClient and its token will be automatically applied to any other services without further use.</p>"},{"location":"tutorials/chunkedgraph/","title":"Chunked Graph","text":"<p>The chunkedgraph is a dynamic oct-tree connected components supervoxel graph.</p> <p></p> <p>A visual representation of an oct-tree (Wikipedia (WhiteTimberwolf) CC BY-SA 3.0)</p> <p>As with any oct-tree, it is organized in hierarchical levels, with the bottom level 1 corresponding to the supervoxels of the segmentations, and the top level being the unique connected components of the supervoxel graph.</p> <p></p> <p>A figure illustrating the spatial chunking and editing of a the chunkedgraph. From Dorkenwald et. al 2021</p> <p>The ChunkedGraph client allows one to interact with the ChunkedGraph service, which stores and updates the supervoxel agglomeration graph. This is most often useful for looking up an object root id of a supervoxel or looking up supervoxels belonging to a root id. The ChunkedGraph client is at <code>client.chunkedgraph</code>.</p>"},{"location":"tutorials/chunkedgraph/#look-up-a-supervoxel","title":"Look up a supervoxel","text":"<p>Usually in Neuroglancer, one never notices supervoxel ids, but they are important for programmatic work. In order to look up the root id for a location in space, one needs to use the supervoxel segmentation to get the associated supervoxel id. The ChunkedGraph client makes this easy using the client.chunkedgraph.get_root_id() method.</p> <pre><code>sv_id = 104200755619042523\nclient.chunkedgraph.get_root_id(supervoxel_id=sv_id)\n</code></pre> <p>However, as proofreading occurs, the root id that a supervoxel belongs to can change. By default, this function returns the current state, however one can also provide a UTC timestamp to get the root id at a particular moment in history. This can be useful for reproducible analysis. Note below that the root id for the same supervoxel is different than it is now.</p> <pre><code>import datetime\n\n# I looked up the UTC POSIX timestamp from a day in early 2019.\ntimestamp = datetime.datetime.utcfromtimestamp(1546595253)\n\nsv_id = 104200755619042523\nclient.chunkedgraph.get_root_id(supervoxel_id=sv_id, timestamp=timestamp)\n</code></pre> <p>If you are doing this across lots of supervoxels (or any nodes) then you can do it more efficiently in one request with client.chunkedgraph.get_roots()</p> <pre><code>node_ids = [104200755619042523, 104200755619042524,104200755619042525]\nroot_ids = client.chunkedgraph.get_roots(node_ids)\n</code></pre>"},{"location":"tutorials/chunkedgraph/#getting-supervoxels-for-a-root-id","title":"Getting supervoxels for a root id","text":"<p>A root id is associated with a particular agglomeration of supervoxels, which can be found with the client.chunkedgraph.get_leaves() method. A new root id is generated for every new change in the chunkedgraph, so time stamps do not apply.</p> <pre><code>root_id = 648518346349541252\nclient.chunkedgraph.get_leaves(root_id)\n</code></pre> <p>You can also query the chunkedgraph not all the way to the bottom, using the stop_layer option</p> <pre><code>root_id = 648518346349541252\nclient.chunkedgraph.get_leaves(root_id,stop_layer=2)\n</code></pre> <p>This will get all the level 2 IDs for this root, which correspond to the lowest chunk of the hierarchy. An analogous option exists for client.chunkedgraph.get_roots(). This is useful to help find nodes to query within the <code>l2cache</code>, amongst other things.</p>"},{"location":"tutorials/chunkedgraph/#other-functions","title":"Other functions","text":"<p>There are a variety of other interesting functions to explore in client.chunkedgraph.</p>"},{"location":"tutorials/framework/","title":"Introduction","text":""},{"location":"tutorials/framework/#caveclient-one-client-for-all-services","title":"CAVEclient: one client for all services","text":"<p>The CAVE Framework consists of a number of different services, each with a specific set of tasks that it can perform through REST endpoints. The CAVEclient is designed to ease programmatic interaction with all of the various endpoints. In addition, most programmatic access requires the use of authentication tokens. In order to collect a given server, datastack name, and user token together into a coherent package that can be used on multiple endpoints, the CAVEclient builds appropriately configured clients for each of the specific services. Each of the individual services has their own specific documentation as well.</p>"},{"location":"tutorials/framework/#global-and-local-services","title":"Global and Local Services","text":"<p>There are two categories of data in CAVE: Global and local. Local services are associated with a single so-called datastack, which refers to a precise collection of imagery and segmentation data that function together. For example, EM imagery and a specific pychunkedgraph segmentation would be one datastack, while the same EM imagery but an initial static segmentation would be another. Datastacks are referred to by a short name, for instance <code>pinky100_public_flat_v185</code>.</p> <p>Global services are those that are potentially shared across multiple different specific datastacks. These include the info service, which can describe the properties of all available datastacks, the authentication service, and the state service that hosts neuroglancer states. Global services are associated with a particular URL (by default <code>http://globalv1.daf-apis.com</code>), but not a single datastack.</p>"},{"location":"tutorials/framework/#initializing-a-caveclient","title":"Initializing a CAVEclient","text":"<p>Assuming that the services are on <code>http://globalv1.daf-apis.com</code> and authentication tokens are either not being used or set up with default values (see Authentication), a simple CAVEclient that can only access global services can be initialized:</p> <pre><code>from caveclient import CAVEclient\n\nclient = CAVEclient()\n</code></pre> <p>Just to confirm that this works, let's see if we can get the EM image source from the InfoService. If you get a list of names of datastacks, all is good. If you have not yet set up an authentication token or you get an authentication error, look at Getting a new token for information about how to set up your auth token.</p> <pre><code>client.info.get_datastacks()\n</code></pre> <p>If you have a specific datastack you want to use, you can initialize your CAVEclient with it. This gives you access to the full range of client functions.</p> <pre><code>client = CAVEclient(datastack_name='my_datastack')\n</code></pre>"},{"location":"tutorials/framework/#using-other-server-addresses","title":"Using Other Server Addresses","text":"<p>If your data is hosted by a different global server, you specify its address when initializing the client.</p> <pre><code>client = CAVEclient(datastack_name='my_datastack', server_address='http://global.myserver.com')\n</code></pre> <p>By default, if you pass both a server address and a datastack, the client will store the mapping from datastack to server address in the same location as the default for authentication tokens. Once stored, the client will automatically use the correct server address for the datastack if none is provided. You can override storing the server address by passing <code>write_server_address=False</code>. Datastacks can be removed from the cache using</p> <p>caveclient.datastack_lookup.reset_server_address_cache(datastack_name).</p>"},{"location":"tutorials/framework/#accessing-specific-clients","title":"Accessing specific clients","text":"<p>Each client can be accessed as a property of the main client. See the documentation at left for the capabilities of each. Assuming your client is named <code>client</code>, the subclients for each service are:</p> <ul> <li>Authentication Service : <code>client.auth</code></li> <li>AnnotationEngine : <code>client.annotation</code></li> <li>PyChunkedGraph : <code>client.chunkedgraph</code></li> <li>InfoService : <code>client.info</code></li> <li>EM Annotation Schemas : <code>client.schemas</code></li> <li>JSON Neuroglancer State Service : <code>client.state</code></li> <li>Skeleton Service : <code>client.skeletonservice</code></li> </ul>"},{"location":"tutorials/info/","title":"Info Service","text":"<p>A datastack has a number of complex paths to various data sources that together comprise a datastack. Rather than hardcode these paths, the InfoService allows one to query the location of each data source. This is also convenient in case data sources change.</p> <p>An InfoClient is accessed at <code>client.info</code>.</p> <pre><code>client = CAVEclient(datastack_name)\nprint(f\"This is an info client for {client.info.datastack_name} on {client.info.server_address}\")\n</code></pre>"},{"location":"tutorials/info/#accessing-datastack-information","title":"Accessing datastack information","text":"<p>All of the information accessible for the datastack can be seen as a dict using <code>get_datastack_info()</code>.</p> <pre><code>info.get_datastack_info()\n</code></pre> <p>Individual entries can be found as well. Use tab autocomplete to see the various possibilities.</p> <pre><code>info.graphene_source()\n</code></pre>"},{"location":"tutorials/info/#adjusting-formatting","title":"Adjusting formatting","text":"<p>Because of the way neuroglancer looks up data versus cloudvolume, sometimes one needs to convert between <code>gs://</code> style paths to <code>https://storage.googleapis.com/</code> stype paths. All of the path sources in the info client accept a <code>format_for</code> argument that can handle this, and correctly adapts to graphene vs precomputed data sources.</p> <pre><code>neuroglancer_style_source = info.image_source(format_for='neuroglancer')\nprint(f\"With gs-style: { neuroglancer_style_source }\")\n\ncloudvolume_style_source = info.image_source(format_for='cloudvolume')\nprint(f\"With https-style: { cloudvolume_style_source }\")\n</code></pre>"},{"location":"tutorials/l2cache/","title":"Level 2 Cache","text":"<p>To understand the level 2 cache, you must understand the structure of the chunkedgraph so see the chunkedgraph tutorial.</p> <p>Nodes on the second level or layer of the graph, corresponds to all the supervoxels that are locally connected to one another within a single level 2 spatial \"chunk\" of the data. The Level 2 Cache, is a service whose job it is to track and update relevant statistics about every level 2 node within the a chunkedgraph. The source code of this service can be found here.</p>"},{"location":"tutorials/l2cache/#finding-level-2-nodes","title":"Finding Level 2 Nodes","text":"<p>The chunkedgraph can be used to find the level2 nodes of a rootID using a <code>stop_layer=2</code> keyword argument on the client.chunkedgraph.get_leaves(). Conversely the level 2 node of a supervoxel can be found using the same keyword argument of client.chunkedgraph.get_roots(). Note if you don't specify a timestamp it will give you the level2 node that is presently associated with the object.</p>"},{"location":"tutorials/l2cache/#statistics","title":"Statistics","text":"<p>The statistics that are available are:</p> <ul> <li>area_nm2: The surface area of the object in square nanometers. Does not include border touching voxels</li> <li>size_nm3: The volume of the object in cubic nanometers,   based on counting voxels in the object.</li> <li>max_dt_nm: The maximum edge distance transform of that   object in nanometers. Meant to capture the   maximum \"thickness\" of the voxels in the   node.</li> <li>mean_dt_nm: The average edge distance transform of that   object in nanometers. Meant to capture the   average \"thickness\" of voxels in that node.</li> <li>rep_coord_nm: A list of x,y,z coordinates in nanometers that   represent a point within the object that is   designed to be close to the \"center\" of the   object. This is the location of the max_dt_nm   value.</li> <li>chunk_intersect_count: A 2 x 3 matrix representing the 6 sides of the   chunk, and whose values represent how many   voxels border that side of the chunk. Meant to   help understand significant the borders with   other chunks are. Ordering is the [[x_bottom, y_bottom, z_bottom],[x_top, y_top, z_top]]   where {xyz}_bottom refers to the face which   has the smallest values for that dimension, and   {xyz}_top refers to the face which has the   largest.</li> <li>pca A 3x3 matrix representing the principal   components of the xyz point cloud of voxels for   this object. Ordering is NxD where N is the   components and D are the xyz dimensions. Meant   to help desribe the orientation of the level 2   chunk. Note that this is not calculated for   very small objects and so might not be present   for all level 2 nodes. You will see that its   availability correlates strongly with size_nm3.</li> <li>pca_val The 3 principal component values for the PCA   components.</li> </ul>"},{"location":"tutorials/l2cache/#retrieving-level-2-statistics","title":"Retrieving Level 2 Statistics","text":"<p>Level 2 stats about nodes can be retreived using the client.l2cache.get_l2data() method. It simply takes a list of level 2 nodes you want to retrieve. Optionally you can specify only the attributes that you are interested in retrieving which will speed up the request.</p>"},{"location":"tutorials/l2cache/#missing-data","title":"Missing Data","text":"<p>The service is constantly watching for changes made to objects and recalculating stats on new level2 nodes that are created, in order to keep its database of statistics current. This however takes some time, and is subject to sporadic rare failures. If you request stats on a level 2 node which are not in the database, you will receive an empty dictionary for that node. This will immediately trigger the system to recalculate the statistics of that missing data, and so it should be available shortly (on the order of seconds) if systems are operational. Please note that PCA is not calculated for very small objects because it is not meaningful. So if you are interested in differentiating whether PCA is not available because it hasn't been calculated, vs when its not available because it is not possible to calculate, you should ask for at least one other non PCA statistic as well. You will see that its availability correlates strongly with <code>size_nm3</code>.</p>"},{"location":"tutorials/l2cache/#use-cases","title":"Use Cases","text":""},{"location":"tutorials/l2cache/#calculate-total-area-and-volume-of-cells","title":"Calculate Total Area and Volume of Cells","text":"<p>Say you want to calculate the total surface area and volume of a object in the dataset. The areas and volume of each component can simply be added together to do this.</p> <pre><code>import pandas as pd\nroot_id = 648518346349541252\nlvl2nodes = client.chunkedgraph.get_leaves(root_id,stop_layer=2)\nl2stats = client.l2cache.get_l2data(lvl2nodes, attributes=['size_nm3','area_nm2'])\nl2df = pd.DataFrame(l2stats).T\ntotal_area_um2=l2df.area_nm2.sum()/(1000*1000)\ntotal_volume_um3 = l2df.size_nm3.sum()/(1000*1000*1000)\n</code></pre> <p>By utilizing the bounds argument of get_leaves, you can also do simple spatially restricted analysis of objects. In fact, because you have data on each level2 node individually, you can segregate the neuron using any labelling of its topology.</p>"},{"location":"tutorials/l2cache/#skeletonization","title":"Skeletonization","text":"<p>Level 2 nodes have \"cross chunk\" edges within the chunkedgraph which represent what level 2 nodes that object is locally connected to. This forms a graph between the level 2 nodes of the object that can be retrieved using the chunkedgraph function client.chunkedgraph. This graph represents a topological representation of the neuron at the resolution of individual chunks, and is guaranteed to be fully connected, unlike a voxel or mesh representation of the neuron which can have gaps where there are defects in the segmentation volume or incorrectly inferred edges at self contact locations.</p> <p>The level 2 graph can be turned into a skeleton representation of a neuron using a graph based TEASAR like algorithm as described for skeletonizing meshes in this MeshParty Documentation. There is an implementation of this approach that utilizes the chunkedgraph and the L2cache if available here and on pypi as <code>pcg-skel</code>. In this implementation the l2cache is used to more accurately place the level 2 nodes in space using the <code>rep_coord_nm</code> value.</p> <p>Note that there is detailed documentation on the Skeleton Client interface at the skeletonization tutorial.</p>"},{"location":"tutorials/l2cache/#trajectory-distributions","title":"Trajectory Distributions","text":"<p>If one is interested in the bulk direction of processes in a region of the brain, one can start with supervoxels in a region, find level 2 nodes that correspond to them, filter out components based on size, (or other criteria such as whether they are part of objects that have components in some other brain area) and look at the distribution of PCA components to understand the directions that those processes are moving within that region of space.</p>"},{"location":"tutorials/materialization/","title":"Materialization","text":"<p>The Materialization client allows one to interact with the materialized annotation tables, that were posted to the annotation service (the annotations tutorial).</p> <p>To see the entire class visit the API doc.</p> <p>The service regularly looks up all annotations and the segids underneath all the boundspatialpoints. You can then query these tables to find out the IDs that underlie the annotations, or the annotations that now intersect with certain IDs.</p> <p>For example, one common pattern is that you have identified a cell based on the location of its cell body, and you have an annotation there.</p> <p>You want to know what are the inputs onto the cell, so you first query the annotation table with your soma annotation, asking for the current ID underneath that soma. Then you query a synapse table for all synapse annotations that have a post-synaptic ID equal to the ID from your soma annotation.</p> <p>In this way your code stays the same, as the proofreading changes and you can track the connectivity of your cell over time.</p>"},{"location":"tutorials/materialization/#initializing-the-client","title":"Initializing the client","text":"<p>By default when you initialize the overall client, it will choose the most recent materialization version available. This may or may not be desirable depending on your use case. If your code involves using specific IDs then you should be using a specific version that is tied to a timepoint where those IDs are valid.</p> <p>To see what versions are available, use the client.materialize.get_versions() function.</p> <pre><code>client.materialize.get_versions()\n</code></pre> <p>Each version has a timestamp it was run on as well as a date when it will expire. You can query all this metadata for a specific version using client.materialize.get_version_metadata() or all versions using client.materialize.get_versions_metadata().</p> <p>To change the default version, alter the <code>.version</code> property of the client. This will change the version for all subsequent calls which expect one, unless you specify a different version in the method call. Note that this also sets the <code>timestamp</code> property of the client to the timestamp of the version for methods which expect a timestamp.</p> <pre><code>client.version = 9\n</code></pre> <p>You can also specify the version when you initialize the client, e.g.:</p> <pre><code>client = CAVEclient('minnie65_public', version=661)\n</code></pre> <p>Or, you can specify the version when making a particular method call.</p>"},{"location":"tutorials/materialization/#browsing-versions","title":"Browsing versions","text":"<p>To see what tables are available in a version you can use client.materialize.get_tables().</p> <p>If you want to read about the description of what that table is, use the annotationengine client client.materialize.get_table_metadata().</p> <p>If you want to read more about the schema for the annotation table use the schema service client.schema.schema_definition().</p> <p>Note, the materialization service has a human readable webpage that links to the other services that might be more convenient for you to browse, to get a link there in ipython display <code>client.materialize.homepage</code></p> <p>for some important tables, the info service has a pointer to which table you should use in the metadata for the datastack. <code>client.info.get_datastack_info()['synapse_table']</code> and <code>client.info.get_datastack_info()['soma_table']</code>.</p> <p>To see how many annotations are in a particular table use</p> <pre><code>nannotations=client.materialize.get_annotation_count('my_table')\n</code></pre>"},{"location":"tutorials/materialization/#querying-tables","title":"Querying tables","text":"<p>To query a small table, you can just download the whole thing using client.materialize.query_table() which will return a dataframe of the table.</p> <p>Note however, some tables, such as the synapse table might be very large 200-300 million rows and the service will only return the first 200,000 results, and not in a deterministic manner. NOTE! This API is not designed to enable enmass downloading of the entire synapse table there are more efficient ways of doing this. Contact your dataset administrator for more information if this is what you are looking to do.</p> <p>To just get a preview, use the limit argument (but note again that this won't be a reproducible set)</p> <pre><code>df=client.materialize.query_table('my_table', limit=10)\n</code></pre> <p>For most applications, you will want to filter the query in some way.</p> <p>We offer seven kinds of filters you can apply: <code>filter_in_dict</code> <code>filter_out_dict</code>, <code>filter_equal_dict</code>, <code>filter_greater_dict</code>, <code>filter_less_dict</code>, <code>filter_greater_equal_dict</code>, and <code>filter_less_equal_dict</code>. For query_table each is specified as a dictionary where the keys are column names, and the values are a list of values (or single value in the case of filter_equal).</p> <p>So for example to query a synapse table for all synapses onto a neuron in flywire you would use</p> <pre><code>synapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.query_table(synapse_table,\n                                  filter_equal_dict = {'post_pt_root_id': MYID})\n</code></pre> <p>The speed of querying is affected by a number of factors, including the size of the data. To improve the performance of results, you can reduce the number of columns returned using <code>select_columns</code>.</p> <p>So for example, if you are only interested in the root_ids and locations of pre_synaptic terminals you might limit the query with select_columns. Also, it is convenient to return the with positions as a column of <code>np.array([x,y,z])</code> coordinates for many purposes. However, sometimes you might prefer to have them split out as separate x, y, z columns. To enable this option use <code>split_columns=True</code>. split_columns=True is faster, as combining them is an extra step.</p> <pre><code>synapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.query_table(synapse_table,\n                                  filter_equal_dict = {'post_pt_root_id': MYID},\n                                  select_columns=['id','pre_pt_root_id', 'pre_pt_position'],\n                                  split_columns=True)\n</code></pre>"},{"location":"tutorials/materialization/#desired-resolution","title":"Desired Resolution","text":"<p>Often you want to have position information in different units. For example, to consider synapse locations or soma locations, you might want to have positions in nanometers or microns.</p> <p>To create neuroglancer views, you might want positions in integer voxels of a size that aligns with the resolution you are used to using Neuroglancer at.</p> <p>Annotation tables can be created and uploaded in varying resolutions according to whatever the user of the table felt was natural. This information is available in the metadata for that table. In addition, you may pass desired_resolution as a keyword argument which will automatically convert all spatial positions into voxels of that size in nanometers.</p> <p>So if you want positions in nanometers, you would pass <code>desired_resolution=[1,1,1]</code>. If you want positions in microns you would pass <code>desired_resolution=[1000,1000,1000]</code>. If you want positions in 4,4,40nm voxel coordinates to use with cloud-volume or neuroglancer you would pass <code>desired_resolution=[4,4,40]</code>.</p>"},{"location":"tutorials/materialization/#spatial-filters","title":"Spatial Filters","text":"<p>You can also filter columns that are associated with spatial locations based upon being within a 3d bounding box.</p> <p>This is done by adding a filter_spatial_dict argument to query_table. The units of the bounding box should be in the units of the voxel_resolution of the table (which can be obtained from client.materialize.get_table_metadata()).</p> <pre><code>bounding_box = [[min_x, min_y, min_z], [max_x, max_y, max_z]]\nsynapse_table = client.info.get_datastack_info('synapse_table')\ndf=client.materialize.query_table(synapse_table,\n                                  filter_equal_dict = {'post_pt_root_id': MYID},\n                                  filter_spatial_dict = {'post_pt_position': bounding_box})\n</code></pre>"},{"location":"tutorials/materialization/#synapse-query","title":"Synapse Query","text":"<p>For synapses in particular, we have a simplified method for querying them with a reduced syntax. client.materialize.synapse_query() lets you specify pre and post synaptic partners as keyword arguments and bounding boxes. The defaults make reasonable assumptions about what you want to query, namely that the synapse_table is the table that the info service advertises, and that if you specify a bounding box, that you want the post_pt_position. These can be overridden of course, but the above bounding box query is simplified to.</p> <p>NOTE! This API is not designed to enable enmass downloading of the entire synapse table there are more efficient ways of doing this. Contact your dataset administrator for more information if this is what you are looking to do.</p> <pre><code>bounding_box = [[min_x, min_y, min_z], [max_x, max_y, max_z]]\ndf=client.materialize.query_table(post_ids = MYID,\n                                  bounding_box=bounding_box)\n</code></pre>"},{"location":"tutorials/materialization/#live-query","title":"Live Query","text":"<p>In order to query the materialized tables above you can only use IDs that were present at the timestamp of the materialization. If you query the tables with an ID that is not valid during the time of the materialization you will get empty results.</p> <p>To check if root_ids are valid at your materialization's timestamp, you can use client.chunkedgraph.is_latest_roots()</p> <pre><code>import numpy as np\nmat_time = client.materialize.get_timestamp()\nis_latest = client.chunkedgraph.is_latest_roots([MYID], timestamp=mat_time)\nassert(np.all(is_latest))\n</code></pre> <p>If you need to lookup what happened to that ID, you can use the chunkedgraph lineage tree, to look into the future or the past, depending on your application you can use client.chunkedgraph.get_lineage_graph().</p> <p>Again, the ideal situation is that you have an annotation in the database which refers to your objects of interest, and querying that table by the id column will return the object in the most recent materialization.</p> <p>However, sometimes you might be browsing and proofreadding the data and get an ID that is more recent that the most recent version available. For convenience, you can use client.materialize.live_query().</p> <p>to automatically update the results of your query to a time in the future, such as now. For example, to pass now, use <code>datetime.datetime.now(datetime.timezone.utc)</code>. Note all timestamps are in UTC throughout the codebase.</p> <pre><code>import datetime\nsynapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.live_query(synapse_table,\n                                  datetime.datetime.now(datetime.timezone.utc),\n                                  filter_equal_dict = {'post_pt_root_id': MYID})\n</code></pre> <p>This will raise an ValueError exception if the IDs passed in your filters are not valid at the timestamp given</p> <p>You can also pass a timestamp directly to query_table and it will call live_query automatically.</p> <pre><code>import datetime\nsynapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.query_table(synapse_table,\n                                  timestamp=datetime.datetime.now(datetime.timezone.utc),\n                                  filter_equal_dict = {'post_pt_root_id': MYID})\n</code></pre> <p>Also, keep in mind if you run multiple queries and at each time pass <code>datetime.datetime.now(datetime.timezone.utc)</code>, there is no guarantee that the IDs will be consistent from query to query, as proofreading might be happening at any time. For larger scale analysis constraining oneself to a materialized version will ensure consistent results.</p> <p>Versions have varying expiration times in order to support the tradeoff between recency and consistency, so before undertaking an analysis project consider what version you want to query and what your plan will be to update your analysis to future versions.</p>"},{"location":"tutorials/materialization/#content-aware-interface-experimental","title":"Content-aware Interface (Experimental)","text":"<p>As of version 5.8.0, we have introduced a new interface to query tables and views. This interface might have small but breaking changes in the near future. :::</p> <p>In order to make the querying interface more consistent across tables, we have introduced an additional alternative interface to filtering and querying data via the <code>client.materialize.tables</code> object. When you instantiate this object, this object finds all of the existing tables and the list of their columns and lets you filter the tables as arguments in the function with suggestions. Moreover, the filtering arguments and the querying arguments are separated into two.</p> <p>Let's see how this works with a simplest example --- downloading a table called <code>nucleus_detection_v0</code>. First, we reference the table as a function and then we run the query --- this is exactly the same as <code>client.materialize.query_table('nucleus_detection_v0')</code>.</p> <pre><code>client = CAVEclient('minnie65_public')\nnuc_df = client.materialize.tables.nucleus_detection_v0().query()\n</code></pre> <p>Where things differ is when we add filters. If we want to query based on a set of values for the field \"id\", for example, we add that as an argument:</p> <pre><code>my_ids = [373879, 111162]\nnuc_df = client.materialize.tables.nucleus_detection_v0(id=my_ids).query()\n</code></pre> <p>Where in this example the <code>id=</code> queries the column <code>id</code> based on the schema. These values can be either individual elements (i.e. an integer or a string) or a list/array of elements, and any field can be used. The tooling will automatically sort out how to format the filtering appropriately when running the query. Importantly, the filtering is identical between querying all types of tables and queries. To see the complete list of fields that can be queried, you can tab-autocomplete or in Jupyter or IPython glance at the docstring with <code>client.materialize.tables.nucleus_detection_v0?</code>.</p> <p>In addition to filtering by one or many values, you can also do spatial queries. To find only annotations within a particular bounding box, you need to specify the spatial point (e.g. <code>pt_position</code>). For each such spatial point, there will be an argument <code>{spatial_point_position}_bbox</code> (e.g. <code>pt_position_bbox</code>) that accepts a 2x3 list of coordinates specifying the upper and lower bounds for the query. For example, to find all nucleus centroids within a particular bounding box, it would be:</p> <pre><code>bounding_box = [[min_x, min_y, min_z], [max_x, max_y, max_z]]\nnuc_df = client.materialize.tables.nucleus_detection_v0(\n    pt_position_bbox=bounding_box\n).query()\n</code></pre> <p>If you are not using any filters, you can omit the parenthesis and use the <code>get_all</code> or <code>get_all_live</code> functions directly, which act similarly to the <code>query</code> and <code>live_query</code> functions respectively. The first example could be rewritten as:</p> <pre><code>nuc_df = client.materialize.tables.nucleus_detection_v0.get_all()\n</code></pre> <p>If you want to list all available fields, you can use the <code>.fields</code> attribute. Similarly, you can get all numeric fields with the <code>.numeric_fields</code> attribute and all spatial fields (allowing bounding box queries) with <code>.spatial_fields</code>.</p> <pre><code>nuc_df = client.materialize.tables.nucleus_detection_v0.spatial_fields\n</code></pre> <p>If you need to specify the table programmatically, you can also use a dictionary-style approach to getting the table filtering function. For example, an equivalent version of the above line would be:</p> <pre><code>my_ids = [373879, 111162]\nmy_table = 'nucleus_detection_v0'\nnuc_df = client.materialize.tables[my_table](id=my_ids).query()\n</code></pre> <p>The <code>query</code> function can also take arguments relating to timestamps or formatting where they act just like in the other query method. In particular, the arguments that apply to <code>query</code> are: <code>select_columns</code>, <code>offset</code>, <code>limit</code>, <code>split_positions</code>, <code>materialization_version</code>, <code>timestamp</code>, <code>metadata</code>, <code>desired_resolution</code>, and <code>get_counts</code>. For example, to add a desired resolution and split positions in the above query, it would look like:</p> <pre><code>my_ids = [373879, 111162]\nnuc_df = client.materialize.tables.nucleus_detection_v0(\n    id=my_ids\n).query(\n    split_positions=True,\n    desired_resolution=[1,1,1],\n)\n</code></pre> <p>Inequalities can also be used in filtering numeric columns. Here, you can pass a dictionary instead of a list of values, with the keys being inequality operators (\"&gt;\", \"&gt;=\", \"&lt;\", and \"&lt;=\") and the values being the comparison. For example, to query for all nuclei with a volume greater than 1000:</p> <pre><code>client.materialize.tables.nucleus_detection_v0(\n    volume={\"&gt;\": 1000}\n).query()\n</code></pre> <p>You can also use multiple inequalities in the same dictionary to filter within a range. For example, to query for all nuclei with a volume between 500 and 750:</p> <pre><code>client.materialize.tables.nucleus_detection_v0(\n    volume={\"&gt;\": 500, \"&lt;\": 750}\n).query()\n</code></pre> <p>If you want to do a live query instead of a materialized query, the filtering remains identical but we use the <code>live_query</code> function instead. The one required argument for <code>live_query</code> is the timestamp.</p> <pre><code>my_ids = [373879, 111162]\nnuc_df = client.materialize.tables.nucleus_detection_v0(\n    id=my_ids\n).live_query(\n    timestamp=datetime.datetime.now(datetime.timezone.utc),\n)\n</code></pre> <p>The live query functions have similar but slightly different arguments: <code>timestamp</code> (required), <code>offset</code>, <code>limit</code>, <code>split_positions</code>, <code>metadata</code>, <code>desired_resolution</code>, and <code>allow_missing_lookups</code>.</p> <p>Note that way that IPython handles docstrings means that while you can use <code>?</code> to get the docstring of the filtering part of the function, you can't simply do something like <code>client.materialize.tables.nucleus_detection_v0().query?</code>. It will tell you the function can't be found, because technically the <code>query</code> function does not yet exist until the table filtering function is called.</p> <p>Instead, if you want to glimpse the docstring of the query or live_query functions, you need to split it into two lines:</p> <pre><code>qry_func = client.materialize.tables.nucleus_detection_v0().query\nqry_func?\n</code></pre> <p>Finally, if the project you are working with has views, a similar interface is available to them via <code>client.materialize.views</code>. Currently views are not compatible with live query, and so only the <code>.query</code> function is available.</p>"},{"location":"tutorials/schemas/","title":"EM Annotation Schemas","text":"<p>The EMAnnotationSchemas client lets one look up the available schemas and how they are defined. This is mostly used for programmatic interactions between services, but can be useful when looking up schema definitions for new tables.</p>"},{"location":"tutorials/schemas/#get-the-list-of-schema","title":"Get the list of schema","text":"<p>One can get the list of all available schema with the <code>schema</code> method. Currently, new schema have to be generated on the server side, although we aim to have a generic set available to use.</p> <pre><code>client.schema.get_schemas()\n</code></pre>"},{"location":"tutorials/schemas/#view-a-specific-schema","title":"View a specific schema","text":"<p>The details of each schema can be viewed with the <code>schema_definition</code> method, formatted as per JSONSchema.</p> <pre><code>example_schema = client.schema.schema_definition('microns_func_coreg')\nexample_schema\n</code></pre> <p>This is mostly useful for programmatic interaction between services at the moment, but can also be used to inspect the expected form of an annotation by digging into the format.</p> <pre><code>example_schema['definitions']['FunctionalCoregistration']\n</code></pre>"},{"location":"tutorials/skeletonization/","title":"Skeletonization","text":"<p>The skeleton service enables you to generate and retrieve skeletons from the server. This saves you the trouble of performing skeletonization routines on your local machine, while also saving you the trouble of understanding many of the details inside of skeletonization. Furthermore, the server offers the ability to generate multiple skeletons simultaneously, in parallel. And lastly, the server caches each skeleton that is generated so that it only ever needs to be generated once. All subsequent requests for the same skeleton will simply return the cached skeleton with notable turn-around-time improvements.</p>"},{"location":"tutorials/skeletonization/#initializing-the-client","title":"Initializing the client","text":"<p>The simplest way to initialize the CAVEclient is by merely providing the datastack of interest:</p> <pre><code>import caveclient as cc\n\nclient = cc.CAVEclient(&lt;datastack_name&gt;)\n</code></pre> <p>With a CAVEclient built, you can now investigate the current build version of the SkeletonClient:</p> <pre><code>client.skeleton.get_version()\n</code></pre> <p>And you can see a list of available skeleton versions. In most cases you will want to use the highest (most recent) version provided.</p> <pre><code>client.skeleton.get_versions()\n</code></pre>"},{"location":"tutorials/skeletonization/#retrieving-a-skeleton","title":"Retrieving a skeleton","text":"<p>Retrieve a skeleton using <code>get_skeleton()</code>. The simplest usage is:</p> <pre><code>sk = client.skeleton.get_skeleton(\n    &lt;root_id&gt;,\n    output_format=&lt;output_format&gt;,\n)\n</code></pre> <p>where the availale output_formats (described below) are:</p> <ul> <li><code>'dict'</code> (default if unspecified)</li> <li><code>'swc'</code> (a Pandas Dataframe)</li> </ul> <p>If the skeleton doesn't exist in the server cache, it may take 20-60 seconds to generate the skeleton before it is returned. This function will block during that time. Any subsequent retrieval of the same skeleton should go very quickly however.</p> <p>To specify a nondefault skeleton version:</p> <pre><code>sk = client.skeleton.get_skeleton(\n    &lt;root_id&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    output_format=&lt;output_format&gt;,\n)\n</code></pre> <p>To specify a specific datastack:</p> <pre><code>sk = client.skeleton.get_skeleton(\n    &lt;root_id&gt;,\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    output_format=&lt;output_format&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#peering-into-the-contents-of-the-cache","title":"Peering into the contents of the cache","text":"<p>Most end-users shouldn't need to use the following function very much, but to see the contents of the cache for a given root id, set of root ids, root id prefix, or set of prefixes:</p> <pre><code>get_cache_contents(\n    root_id_prefixes=&lt;root_id&gt;,\n)\n</code></pre> <p>You can also add additional parameters as needed:</p> <pre><code>get_cache_contents(\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    root_id_prefixes=&lt;root_id&gt;,\n)\n</code></pre> <p>The primary parameter, <code>root_id_prefixes</code>, can be a list of root ids:</p> <pre><code>get_cache_contents(\n    root_id_prefixes=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>The primary parameter can also be a root id prefix, which will match any associated root ids. Since this could potentially return a large list of results, there is an optional <code>limit</code> parameter so you don't overwhelm the memory of your processing environment, e.g., a Jupyter notebook or some other Python script running on your local machine:</p> <pre><code>get_cache_contents(\n    root_id_prefixes=&lt;root_id_prefix&gt;,\n    limit=&lt;limit&gt;,\n)\n</code></pre> <p>Note that the limit only constraints the size of the return value. The internal operation of the function will still receive the full list when it passes the prefix on to CloudFiles. Consequently, calling this function for a short prefix may block for a long time.</p> <p>And of course you can also pass in a list of prefixes (or a mixture of full ids and partial prefixes):</p> <pre><code>get_cache_contents(\n    root_id_prefixes=[&lt;root_id_prefix&gt;, &lt;root_id_prefix&gt;, ...],\n    limit=&lt;limit&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#querying-the-presence-of-a-skeleton-in-the-cache","title":"Querying the presence of a skeleton in the cache","text":"<p>The function shown above isn't necessarily the most direct way to simply inquire whether a skeleton exists in the cache for a given root id. For that purpose, the following function is better suited:</p> <pre><code>skeletons_exist(\n    root_ids=&lt;root_id&gt;,\n)\n</code></pre> <p>Or:</p> <pre><code>skeletons_exist(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>Note that this function doesn't accept prefixes, as supported by <code>cache_query_contents()</code>. Only full root ides are supported. When querying with as single root id, the return value will be a boolean. When querying with a list of ids, the return value will be a Python dictionary mapping from each id to a boolean.</p> <p>This function also takes the same optional parameters described above:</p> <pre><code>skeletons_exist(\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    root_ids=&lt;root_id&gt;,  # Or [&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#retrieving-multiple-skeletons","title":"Retrieving multiple skeletons","text":"<p>You can retrieve a large set of skeletons in a single function call:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>If any skeletons are not generated yet, the default behavior is to skip those root ids and only return skeletons that are already available. But you can override this default behavior:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    generate_missing_skeletons=[False|True],\n)\n</code></pre> <p>Any root ids for which skeletonization is required will be generated one at a time, at a cost of 20-60 seconds each. Consequently, there is a hard-coded limit of 10, after which all subsequent missing skeletons will not be returned.</p> <p>By default, skeletons are returned in JSON format. However SWC is also supported, thusly:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    output_format=&lt;\"json\"|\"swc\"&gt;\n)\n</code></pre> <p>And the usual defaults can be overridden again:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#generating-multiple-skeletons-in-parallel","title":"Generating multiple skeletons in parallel","text":"<p><code>get_bulk_skeletons()</code> is not an effective way to produce a large number of skeletons since it operates synchronously, generating one skeleton at a time. In order to generate a large number of skeletons it is better to do so in parallel. The following function dispatches numerous root ids for skeletonization without returning anything immediately. The root ids are then distributed on the server for parallel skeletonization and eventual caching. Once they are in the cache, you can retrieve them. Of course, it can be tricky to know when they are available. That is addressed further below. Here's how to dispatch asynchronous bulk skeletonization:</p> <pre><code>generate_bulk_skeletons_async(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>And with the usual overrides:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#retrieving-asynchronously-generated-skeletons","title":"Retrieving asynchronously generated skeletons","text":"<p>In order to retrieve asynchronously generated skeletons, it is necessary to poll the cache for the availability of the skeletons and then eventually retrieve them. Here's an example of such a workflow:</p> <pre><code># Dispatch multiple asynchronous, parallel skeletonization and caching processes\ngenerate_bulk_skeletons_async(root_ids)\n\n# Repeatedly query the cache for the existence of the skeletons until they are all available\nwhile True:\n    skeletons_that_exist = client.skeleton.skeletons_exist(root_ids=rids)\n    num_skeletons_found = sum(skeletons_that_exist.values())\n    if num_skeletons_found == len(rids):\n        break\n    sleep(10)  # Pause for ten seconds and check again\n\n# Retrieve the skeletons (remember, SWC is also offered)\nskeletons_json = get_bulk_skeletons(root_ids)\n</code></pre>"},{"location":"tutorials/state/","title":"JSON Neuroglancer State Service","text":"<p>We store the JSON description of a Neuroglancer state in a simple database at the JSON Service. This is a convenient way to build states to distribute to people, or pull states to parse work by individuals. The JSON Client is at <code>client.state</code></p> <pre><code>client.state\n</code></pre>"},{"location":"tutorials/state/#retrieving-a-state","title":"Retrieving a state","text":"<p>JSON states are found simply by their ID, which you get when uploading a state. You can download a state with <code>get_state_json</code>.</p> <pre><code>example_id = 4845531975188480\nexample_state = client.state.get_state_json(test_id)\nexample_state['layers'][0]\n</code></pre>"},{"location":"tutorials/state/#uploading-a-state","title":"Uploading a state","text":"<p>You can also upload states with <code>upload_state_json</code>. If you do this, the state id is returned by the function. Note that there is no easy way to query what you uploaded later, so be VERY CAREFUL with this state id if you wish to see it again.</p> <p>Note: If you are working with a Neuroglancer Viewer object or similar, in order to upload, use viewer.state.to_json() to generate this representation.</p> <pre><code>example_state['layers'][0]['name'] = 'example_name'\nnew_id = client.state.upload_state_json(example_state)\n</code></pre> <pre><code>test_state = client.state.get_state_json(new_id)\ntest_state['layers'][0]['name']\n</code></pre>"},{"location":"tutorials/state/#generating-a-neuroglancer-url","title":"Generating a Neuroglancer URL","text":"<p>Once you have a state ID, you want to turn it into a well-formatted link. So you don\\'t have to remember all the endpoints, we can do this from the state client.</p> <pre><code>ngl_base = 'neuromancer-seung-import.appspot.com'\nclient.state.build_neuroglancer_url(new_id, ngl_base)\n</code></pre> <p>Note that the neuroglancer base can be found in the info service under <code>client.info.viewer_site()</code>.</p>"}]}
\ No newline at end of file
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"<p>CAVE is short for Connectome Annotation Versioning Engine. CAVE is a set of microservices that provide a framework for storing and versioning connectomics data and large sets of dynamic annotations, metadata, and segmentations. This repository supplies client-side code to easily interact with the microservices in CAVE.</p>"},{"location":"#documentation-layout","title":"Documentation Layout","text":"<p>To learn how to install <code>caveclient</code>, visit Installation.</p> <p>To see hands-on examples of using <code>caveclient</code> in a notebook, visit the Tutorials.</p> <p>To see API reference documentation for interacting with a <code>caveclient.CAVEclient</code> object, visit the Client API.</p> <p>To see a glossary of terms used in the documentation, visit the Glossary (work in progress).</p> <p>For information about how to contribute to the documentation or the package, visit the Contributing page. Feedback on the documentation is welcome! Please open an issue or use the \"Edit this page\" button at the top right of any page to suggest changes.</p> <p>To read a full description of the Connectome Annotation Versioning Engine, including a description of the various services please read this paper.</p>"},{"location":"changelog/","title":"Changelog","text":""},{"location":"changelog/#741-december-17-2024","title":"7.4.1 (December 17, 2024)","text":"<ul> <li>Enabled skeleton client skeleton_exists() and generate_bulk_skeletons_async() endpoints to accept a either a list or a numpy array of root ids.</li> <li>Eliminated skeleton client generate_bulk_skeletons_async() issue in which passing a large number of root ids would cause a \"URI too big\" error.</li> </ul>"},{"location":"changelog/#740-december-12-2024","title":"7.4.0 (December 12, 2024)","text":"<p>annotation module: Added support for \"None\", \"none\", etc. to remove notice_text and  improved documentation on method</p>"},{"location":"changelog/#732-december-10-2024","title":"7.3.2 (December 10, 2024)","text":"<p>Dropped oldest supported numpy requirements and declared python 3.8 support</p>"},{"location":"changelog/#731-december-5-2024","title":"7.3.1 (December 5, 2024)","text":"<p>Improved documentation to annotation.update_table_metadata</p>"},{"location":"changelog/#730-december-5-2024","title":"7.3.0 (December 5, 2024)","text":"<p>Added get_minimal_covering_nodes to chunkedgraph (#274)</p>"},{"location":"changelog/#721-december-3-2024","title":"7.2.1 (December 3, 2024)","text":"<p>Added raw strings to avoid deprecation warning (#275)</p>"},{"location":"changelog/#720-december-3-2024","title":"7.2.0 (December 3, 2024)","text":"<p>Allow get_state_json to accept direct URLs (#271)</p>"},{"location":"changelog/#710-december-3-2024","title":"7.1.0 (December 3, 2024)","text":"<p>Added get_leaves_multiple to chunkedgraph module (#273)</p>"},{"location":"changelog/#700-november-20-2024","title":"7.0.0 (November 20, 2024)","text":"<ul> <li>Simplified skeleton client interface to only accept formats of 'dict' and 'swc'   Structure of the dictionary returned was also changed compared to previous \"json\" format, where the dict is flat with only one level of keys.   Corresponding change made to MeshParty 1.18.0 allows for hydration of MeshParty Skeleton object from this dict format. </li> </ul>"},{"location":"changelog/#650-november-15-2024","title":"6.5.0 (November 15, 2024)","text":"<ul> <li>Added endpoints for bulk skeleton retrieval and querying whether a skeleton exists or not.</li> </ul>"},{"location":"changelog/#641","title":"6.4.1","text":"<ul> <li>Fix networkx deprecation warning.</li> </ul>"},{"location":"changelog/#640-october-31-2024","title":"6.4.0 (October 31, 2024)","text":"<ul> <li>Added a CAVEclientMock function to ease the use of fully mocked CAVEclients in testing.</li> <li>Fix a bug about use of global server.</li> <li>Fix writing of local server secret to be more secure.</li> </ul>"},{"location":"changelog/#630","title":"6.3.0","text":"<ul> <li>Add a convenience function to get a complete L2 feature dataframe.</li> </ul>"},{"location":"changelog/#620","title":"6.2.0","text":"<ul> <li>Fix things so that auth will not error if the secrets directory is read-only.</li> </ul>"},{"location":"changelog/#612","title":"6.1.2","text":"<ul> <li>Fix query filter bug in tables interface.</li> </ul>"},{"location":"changelog/#611","title":"6.1.1","text":"<ul> <li>Fix warnings.</li> </ul>"},{"location":"changelog/#610","title":"6.1.0","text":"<ul> <li>Added support for inequality filters for numerical columns in materialization tables interface.</li> </ul>"},{"location":"changelog/#600-october-14-2024","title":"6.0.0 (October 14, 2024)","text":"<ul> <li>Refactored CAVEclient internals away from a factory syntax. Most users should not notice a difference, but code that relied on specific subclient type logic may be affected.</li> </ul>"},{"location":"changelog/#5250","title":"5.25.0","text":"<ul> <li>Added ability to suppress table warnings in client.materialize.query_table and similar methods by setting <code>log_warnings=False</code></li> </ul>"},{"location":"changelog/#5200-april-8-2024","title":"5.20.0 (April 8, 2024)","text":"<ul> <li>Added generalized support for detecting server versions to provide timely exceptions to users</li> <li>Used new support to check that chunkegraph has updated version before using spatial bounds kwarg   on client.chunkedgraph.level2_chunk_graph</li> <li>Added support for postign and getting segment properties files to client.state</li> </ul>"},{"location":"changelog/#5180","title":"5.18.0","text":"<ul> <li>Added serialization support for pandas.index</li> </ul>"},{"location":"changelog/#5173","title":"5.17.3","text":"<ul> <li>Minor documentation typo fix</li> </ul>"},{"location":"changelog/#5172","title":"5.17.2","text":"<ul> <li>Bug fixes related to table_manager interface</li> </ul>"},{"location":"changelog/#5171","title":"5.17.1","text":"<ul> <li>Bug fixes related to table_manager interface</li> </ul>"},{"location":"changelog/#5170","title":"5.17.0","text":"<ul> <li>Fix attrs in dataframe attributes of client.materialize results to remove numpy arrays to allow concatenation of dataframes</li> <li>Added getting multiple schemas in one call to improve initialization of table_manager interface of materialization</li> </ul>"},{"location":"changelog/#5161","title":"5.16.1","text":"<ul> <li>Bugfix on client.chunkedgrpah.level2_chunk_graph</li> </ul>"},{"location":"changelog/#5160","title":"5.16.0","text":"<ul> <li>Added bounding box query to client.chunkedgraph.level2_chunk_graph</li> <li>Fix default materialization version client when server not advertising correctly</li> </ul>"},{"location":"changelog/#5151-jan-18-2024","title":"5.15.1 (Jan 18, 2024)","text":"<ul> <li>minor improvements to release process</li> </ul>"},{"location":"changelog/#5150-jan-18-2024","title":"5.15.0 (Jan 18, 2024)","text":"<ul> <li>Improved documentation with types</li> <li>Improved testing on more python versions</li> <li>Bugfixes for pyton 3.12 compatability</li> </ul>"},{"location":"changelog/#5140-november-24-2023","title":"5.14.0 (November 24, 2023)","text":"<ul> <li>Made automatic detection of neuroglancer versioning when constructing link shortener links</li> </ul>"},{"location":"changelog/#5130-october-26-2023","title":"5.13.0 (October 26, 2023)","text":"<ul> <li>Add option to get expired versions to client.materialize.get_versions</li> </ul>"},{"location":"changelog/#5121-october-16-2023","title":"5.12.1 (October 16, 2023)","text":"<ul> <li>Bugfixes for client.chunkedgraph.get_latest_roots</li> </ul>"},{"location":"changelog/#5120-october-16-2023","title":"5.12.0 (October 16, 2023)","text":"<ul> <li>Improved logic for client.chunkedgraph.get_latest_roots to work forward or backwards in time</li> </ul>"},{"location":"changelog/#5110-september-19-2023","title":"5.11.0 (September 19, 2023)","text":"<ul> <li>Added filter_regex_dict options to client.materialize.query_table interface</li> </ul>"},{"location":"changelog/#5102-august-162023","title":"5.10.2 (August 16,2023)","text":"<ul> <li>Fixed pyarrow support for live_live query</li> </ul>"},{"location":"changelog/#5101-august-142023","title":"5.10.1 (August 14,2023)","text":"<ul> <li>Changed random_sample argument to be an integer number of annotations rather than a floating fraction of table</li> <li>Added option to live_query</li> </ul>"},{"location":"changelog/#590-august-14-2023","title":"5.9.0 (August 14, 2023)","text":"<ul> <li>Added support for native pyarrow deserialization, allowing upgrade to pyarrow version</li> </ul>"},{"location":"changelog/#580","title":"5.8.0","text":"<ul> <li>Allowed int64 root ids to serialize properly</li> <li>Added warning that client.materialize.tables interface is in beta</li> </ul>"},{"location":"changelog/#570","title":"5.7.0","text":"<ul> <li>Fix to ensure stop_layer is at least 1</li> <li>Added client.chunkedgraph.suggest_latest_roots</li> </ul>"},{"location":"changelog/#560","title":"5.6.0","text":"<ul> <li>Added views to client.materialize.tables interface</li> <li>Added optional argument to allow invalid root ids when querying live live, versus creating an exception</li> </ul>"},{"location":"changelog/#551","title":"5.5.1","text":"<ul> <li>documentation fixes on client.materialize.join_query</li> </ul>"},{"location":"changelog/#550","title":"5.5.0","text":"<ul> <li>added methods for different neuroglancer state formats to client.state.</li> </ul>"},{"location":"changelog/#543","title":"5.4.3","text":"<ul> <li>Added 'view' querying options to materialization</li> <li>Added client.materialize.tables interface</li> <li>Added client.materialize.get_tables_metadata to get all metadata in one call</li> </ul>"},{"location":"changelog/#520","title":"5.2.0","text":"<ul> <li>Added local caching of datastack names &gt; server_address to simplify initialization of clients   with servers other than global.daf-apis.com.</li> </ul> <p>Cache is saved on a local file ~/.cloudvolume/secrets/cave_datastack_to_server_map.json</p> <p>Cache will populate the first time caveclient.CAVEclient('my_datastack', server_address=\"https://my_server.com\") is called. Subsequent calls can then just be caveclient.CAVEclient('my_datastack').</p>"},{"location":"changelog/#510","title":"5.1.0","text":"<ul> <li>Added get_oldest_timestamp call to chunkedgraph</li> </ul>"},{"location":"changelog/#501","title":"5.0.1","text":"<ul> <li>Fixed bug with desired_resolution being set at the client level   was being ignored in &gt;5.0.0</li> </ul>"},{"location":"changelog/#500","title":"5.0.0","text":"<ul> <li>Added support for the new CAVE Materialization 3.0 API   Includes support for the new materialization API, which allows for   server side conversion of the units of position, and ensures that   all positions are returned with the same units, even after joins.</li> <li>Added support for querying databases that were materialized without merging   tables together. This will allow for faster materializations.</li> <li>Removed support for LiveLive query from the Materialization 2.0 API client.   Note.. &lt;5.0.0 clients interacting with MaterializationEngine &gt;4.7.0 servers will   use live live query but will doubly convert the units of position if you ask   for a desired resolution, as the old client will also do a conversion server side.</li> <li>Fixed interaction with api version querying of servers from individual   clients to work with verify=False. (useful for testing)</li> <li>Stored infromation from client about mapping between dataframe and table names   and original column names.</li> <li>Added support for suffixes and select columns to be passed by dictionary rather than list   making the selection an application of suffixes more explicit when there are collisions   between column names in joined tables.</li> </ul>"},{"location":"changelog/#older-upgrade-notes","title":"Older Upgrade Notes","text":"<p>Change all select_column calls to pass dictionaries rather than lists. Change all suffix calls to pass dictionaries rather than lists. Advocate for your server administrator to upgrade to MaterializationEngine 4.7.0 or later, so you can use the new MaterializationEngine 3.0 API and client.</p>"},{"location":"contributing/","title":"Contributing","text":"<p>Contributions are welcome, and they are greatly appreciated! Every little bit helps, and credit will always be given.</p> <p>You can contribute in many ways:</p>"},{"location":"contributing/#types-of-contributions","title":"Types of Contributions","text":""},{"location":"contributing/#report-bugs","title":"Report Bugs","text":"<p>Report bugs to our issues page.</p> <p>If you are reporting a bug, please include:</p> <ul> <li>Your operating system name and version.</li> <li>Any details about your local setup that might be helpful in troubleshooting.</li> <li>Detailed steps to reproduce the bug, in the form of a minimal reproducible example.</li> </ul>"},{"location":"contributing/#fix-bugs","title":"Fix Bugs","text":"<p>Look through the GitHub issues for bugs. Anything tagged with \"bug\" and \"help wanted\" is open to whoever wants to implement it.</p>"},{"location":"contributing/#implement-features","title":"Implement Features","text":"<p>Look through the GitHub issues for features. Anything tagged with \"enhancement\" and \"help wanted\" is open to whoever wants to implement it.</p>"},{"location":"contributing/#write-documentation","title":"Write Documentation","text":"<p><code>caveclient</code> could always use more documentation, whether as part of the official <code>caveclient</code> docs, in docstrings, or even on the web in blog posts, articles, and such.</p>"},{"location":"contributing/#submit-feedback","title":"Submit Feedback","text":"<p>The best way to send feedback is to create an issue on GitHub.</p> <p>If you are proposing a feature:</p> <ul> <li>Explain in detail how it would work.</li> <li>Keep the scope as narrow as possible, to make it easier to implement.</li> <li>Remember that while contributions are welcome, developer/maintainer time is limited.</li> </ul>"},{"location":"contributing/#modifying-code","title":"Modifying Code","text":"<p>Ready to contribute? Here's how to set up <code>caveclient</code> for local development.</p> <ul> <li>Fork the repo on GitHub.</li> <li>Clone your fork locally</li> </ul> <pre><code>git clone git@github.com:your_name_here/CAVEclient.git\n</code></pre> <ul> <li>We use <code>uv</code> for various developer tasks. Ensure you have <code>uv</code> installed according to the installation instructions.</li> </ul> <p>Note</p> <p>While we recommend using <code>uv</code> as described here, these tasks could also be achieved using <code>pip</code> to install and run the various required tools. You can view the development requirements and build/check commands in our <code>pyproject.toml</code>, so we avoid duplicating them here.</p> <ul> <li>Navigate to the newly cloned directory, e.g.:</li> </ul> <pre><code>cd CAVEclient\n</code></pre> <ul> <li>Create a synced virtual environment, optionally specifying a Python version:</li> </ul> <pre><code>uv sync --python 3.12\n</code></pre> <ul> <li>Create a branch for local development:</li> </ul> <pre><code>git checkout -b name-of-your-bugfix-or-feature\n</code></pre> <ul> <li> <p>Make your changes locally</p> </li> <li> <p>If you have added code that should be tested, add tests.</p> </li> <li> <p>If you have modified dependencies in any way, make sure to run</p> </li> </ul> <pre><code>uv sync\n</code></pre> <ul> <li>Make sure you have added documentation for any additions or modifications to public functions or classes. You can build the documentation locally to make sure it renders correctly with:</li> </ul> <pre><code>uvx --from poethepoet poe doc-build\n</code></pre>"},{"location":"contributing/#automated-checks","title":"Automated Checks","text":"<ul> <li>Run the autoformatter:</li> </ul> <pre><code>uvx --from poethepoet poe lint-fix\n</code></pre> <ul> <li>Ensure that your changes pass the checks that will be run on Github Actions, including building the documentation, checking the formatting of the code, and running the tests. To run all at once, do:</li> </ul> <pre><code>uvx --from poethepoet poe checks\n</code></pre> <ul> <li> <p>You may be interested in running some of these checks individually, such as:</p> <ul> <li>To run the tests:</li> </ul> <pre><code>uvx --from poethepoet poe test\n</code></pre> <ul> <li>To build the documentation:</li> </ul> <pre><code>uvx --from poethepoet poe doc-build\n</code></pre> <ul> <li>To run the linter</li> </ul> <pre><code>uvx --from poethepoet poe lint\n</code></pre> </li> </ul>"},{"location":"contributing/#submitting-a-pull-request","title":"Submitting a Pull Request","text":"<ul> <li>Ensure your code has passed all of the tests described above.</li> <li>Commit your changes and push your branch to GitHub:</li> </ul> <pre><code>git add .\ngit commit -m \"Your detailed description of your changes.\"\ngit push origin name-of-your-bugfix-or-feature\n</code></pre> <ul> <li>Submit a pull request through the GitHub website.</li> </ul> <p>Before you submit a pull request, check that it meets these guidelines:</p> <ul> <li>The pull request should include tests if adding a new feature.</li> <li> <p>The docs should be updated with whatever changes you have made. Put   your new functionality into a function with a docstring, and make sure the new   functionality is documented after building the documentation (described above).</p> </li> <li> <p>Once you submit a pull request, automated checks will run. You may require administrator approval before running these checks if this is your first time contributing to the repo.</p> </li> </ul>"},{"location":"glossary/","title":"Glossary","text":"<p>Warning</p> <p>This glossary is a work in progress; for now we are documenting the commonly used  terms that we need to define. Please feel free to contribute definitions or  additional terms.</p> <ul> <li>Datastack</li> <li>Voxel resolution</li> <li>Segmentation resolution</li> <li>MIP</li> <li>Segmentation</li> <li>View</li> </ul>"},{"location":"installation/","title":"Installation","text":""},{"location":"installation/#stable-release","title":"Stable release","text":"<p>To install <code>caveclient</code>, run this command in your terminal:</p> <pre><code>pip install caveclient\n</code></pre> <p>This is the preferred method to install <code>caveclient</code>, as it will always install the most recent stable release.</p> <p>You can also specify a particular version, e.g.</p> <pre><code>pip install caveclient==5.0.0\n</code></pre> <p>If you don't have pip installed, this Python installation guide can guide you through the process.</p>"},{"location":"installation/#from-source","title":"From source","text":"<p>The source for <code>caveclient</code> can be downloaded from the Github repo.</p> <p>You can either clone the public repository:</p> <pre><code>git clone git://github.com/CAVEconnectome/CAVEclient\n</code></pre> <p>Or download the tarball:</p> <pre><code>curl -OJL https://github.com/CAVEconnectome/CAVEclient/tarball/master\n</code></pre> <p>Once you have a copy of the source, you can install it with:</p> <pre><code>pip install .\n</code></pre> <p>Or in editable mode, it can be installed with:</p> <pre><code>pip install -e .\n</code></pre>"},{"location":"api/","title":"Overview","text":"<p>The most common method of interacting with the CAVE Framework is by instantiating a client (<code>caveclient.CAVEclient</code>) and then using that client to interact with various services. Under the hood, the CAVEclient is a collection of individual clients, which can be accessed via properties. For example, to access the materialization client, you can use <code>client.materialize</code>, which (up to the exact version) will actually return a <code>MaterializationClient</code> object.</p> <p>These pages describe the functionality of each of the individual clients, but note that some features may not be available if the server you are accessing is using older versions of particular services.</p>"},{"location":"api/annotation/","title":"client.annotation","text":"<p>Client for interacting with the annotation engine.</p> <p>Methods:</p> Name Description <code>create_table</code> <p>Creates a new data table based on an existing schema</p> <code>delete_annotation</code> <p>Delete one or more annotations in a table. Annotations that are</p> <code>delete_table</code> <p>Marks a table for deletion requires super admin privileges</p> <code>get_annotation</code> <p>Retrieve an annotation or annotations by id(s) and table name.</p> <code>get_annotation_count</code> <p>Get number of annotations in a table</p> <code>get_table_metadata</code> <p>Get metadata about a table</p> <code>get_tables</code> <p>Gets a list of table names for a aligned_volume_name</p> <code>post_annotation</code> <p>Post one or more new annotations to a table in the AnnotationEngine.</p> <code>post_annotation_df</code> <p>Post one or more new annotations to a table in the AnnotationEngine.</p> <code>process_position_columns</code> <p>Process a dataframe into a list of dictionaries</p> <code>raise_for_status</code> <p>Raises requests.HTTPError, if one occurred.</p> <code>stage_annotations</code> <p>Get a StagedAnnotations object to help produce correctly formatted annotations for a given table or schema.</p> <code>update_annotation</code> <p>Update one or more new annotations to a table in the AnnotationEngine.</p> <code>update_annotation_df</code> <p>Update one or more annotations to a table in the AnnotationEngine using a</p> <code>update_metadata</code> <p>Update the metadata on an existing table</p> <code>upload_staged_annotations</code> <p>Upload annotations directly from an Annotation Guide object.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.create_table","title":"<code>create_table(table_name, schema_name, description, voxel_resolution, reference_table=None, track_target_id_updates=None, flat_segmentation_source=None, user_id=None, aligned_volume_name=None, write_permission='PRIVATE', read_permission='PUBLIC', notice_text=None)</code>","text":"<p>Creates a new data table based on an existing schema</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the new table. Cannot be the same as an existing table</p> required <code>schema_name</code> <code>str</code> <p>Name of the schema for the new table.</p> required <code>description</code> <code>str</code> <p>Human readable description for what is in the table. Should include information about who generated the table What data it covers, and how it should be interpreted. And who should you talk to if you want to use it. An Example: a manual synapse table to detect chandelier synapses on 81 PyC cells with complete AISs [created by Agnes - agnesb@alleninstitute.org, uploaded by Forrest]</p> required <code>voxel_resolution</code> <code>List[float]</code> <p>voxel resolution points will be uploaded in, typically nm, i.e [1,1,1] means nanometers [4,4,40] would be 4nm, 4nm, 40nm voxels</p> required <code>reference_table</code> <code>str</code> <p>If the schema you are using is a reference schema Meaning it is an annotation of another annotation. Then you need to specify what the target table those annotations are in.</p> <code>None</code> <code>track_target_id_updates</code> <code>bool</code> <p>Indicates whether to automatically update reference table's foreign key if target annotation table row is updated.</p> <code>None</code> <code>flat_segmentation_source</code> <code>str</code> <p>the source to a flat segmentation that corresponds to this table i.e. precomputed:\\gs:\\mybucket      his_tables_annotation</p> <code>None</code> <code>user_id</code> <code>int</code> <p>If you are uploading this schema on someone else's behalf and you want to link this table with their ID, you can specify it here Otherwise, the table will be created with your userID in the user_id column.</p> <code>None</code> <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <code>write_permission</code> <code>str</code> <p>What permissions to give the table for writing.  One of PRIVATE: only you can write to this table (DEFAULT) GROUP: only members that share a group with you can write (excluding some groups) PUBLIC: Anyone can write to this table. Note all data is logged, and deletes are done by marking rows as deleted, so all data is always recoverable</p> <code>'PRIVATE'</code> <code>read_permission</code> <code>str</code> <p>What permissions to give the table for reading. One of PRIVATE: only you can read this table. Intended to be used for sorting out bugs. GROUP: only members that share a group with you can read (intended for within group vetting) PUBLIC: anyone with permissions to read this datastack can read this data (DEFAULT)</p> <code>'PUBLIC'</code> <code>notice_text</code> <code>str</code> <p>Text the user will see when querying this table. Can be used to warn users of flaws, and uncertainty in the data, or to advertise citations that should be used with this table. Defaults to None, no text. If you want to remove text, send empty string.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p> <p>Examples:</p> <p>Basic annotation table:</p> <pre><code>description = \"Some description about the table\"\nvoxel_res = [4,4,40]\nclient.create_table(\"some_synapse_table\", \"synapse\", description, voxel_res)\n</code></pre>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.delete_annotation","title":"<code>delete_annotation(table_name, annotation_ids, aligned_volume_name=None)</code>","text":"<p>Delete one or more annotations in a table. Annotations that are deleted are recorded as 'non-valid' but are not physically removed from the table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>annotation_ids</code> <code>(dict or list)</code> <p>A list of (or a single) dict of schematized annotation data matching the target table. each dict must contain an \"id\" field which is the ID of the annotation to update</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON: a list of new annotation IDs.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.delete_table","title":"<code>delete_table(table_name, aligned_volume_name=None)</code>","text":"<p>Marks a table for deletion requires super admin privileges</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_annotation","title":"<code>get_annotation(table_name, annotation_ids, aligned_volume_name=None)</code>","text":"<p>Retrieve an annotation or annotations by id(s) and table name.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table</p> required <code>annotation_ids</code> <code>int or iterable</code> <p>ID or IDS of the annotation to retreive</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>Annotation data</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_annotation_count","title":"<code>get_annotation_count(table_name, aligned_volume_name=None)</code>","text":"<p>Get number of annotations in a table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>int</code> <p>number of annotations</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_table_metadata","title":"<code>get_table_metadata(table_name, aligned_volume_name=None)</code>","text":"<p>Get metadata about a table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>aligned_volume_name</code> <code>str</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>metadata about table</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.get_tables","title":"<code>get_tables(aligned_volume_name=None)</code>","text":"<p>Gets a list of table names for a aligned_volume_name</p> <p>Parameters:</p> Name Type Description Default <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>List of table names</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.post_annotation","title":"<code>post_annotation(table_name, data, aligned_volume_name=None)</code>","text":"<p>Post one or more new annotations to a table in the AnnotationEngine. All inserted annotations will be marked as 'valid'. To invalidate annotations refer to 'update_annotation', 'update_annotation_df' and 'delete_annotation' methods.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>data</code> <code>(dict or list)</code> <p>A list of (or a single) dict of schematized annotation data matching the target table.</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.post_annotation_df","title":"<code>post_annotation_df(table_name, df, position_columns, aligned_volume_name=None)</code>","text":"<p>Post one or more new annotations to a table in the AnnotationEngine. All inserted annotations will be marked as 'valid'. To invalidate annotations see 'update_annotation', 'update_annotation_df' and 'delete_annotation' methods.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>df</code> <code>DataFrame</code> <p>A pandas dataframe containing the annotations. Columns should be fields in schema, position columns need to be called out in position_columns argument.</p> required <code>position_columns</code> <code>Optional[Union[Iterable[str], Mapping[str, str]]]</code> <p>if None, will look for all columns with 'X_position' in the name and assume they go in fields called \"X\". if Iterable assumes each column given ends in _position. (i.e. ['pt_position'] if 'pt' is the name of the position field in schema) if Mapping, keys are names of columns in dataframe, values are the names of the fields (i.e. {'pt_column': 'pt'} would be correct if you had one column named 'pt_column' which needed to go into a schema with a position column called 'pt')</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.process_position_columns","title":"<code>process_position_columns(df, position_columns)</code>  <code>staticmethod</code>","text":"<p>Process a dataframe into a list of dictionaries</p> <p>Parameters:</p> Name Type Description Default <code>df</code> <code>DataFrame</code> <p>Dataframe to process</p> required <code>position_columns</code> <code>Optional[Union[Iterable[str], Mapping[str, str]]]</code> <p>See <code>.post_annotation_df</code></p> required <p>Returns:</p> Type Description <code>dict</code> <p>Annotations ready for posting</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.raise_for_status","title":"<code>raise_for_status(r, log_warning=True)</code>  <code>staticmethod</code>","text":"<p>Raises requests.HTTPError, if one occurred.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.stage_annotations","title":"<code>stage_annotations(table_name=None, schema_name=None, update=False, id_field=False, table_resolution=None, annotation_resolution=None)</code>","text":"<p>Get a StagedAnnotations object to help produce correctly formatted annotations for a given table or schema. StagedAnnotation objects can be uploaded directly with <code>upload_staged_annotations</code>.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Table name to stage annotations for, by default None.</p> <code>None</code> <code>schema_name</code> <code>str</code> <p>Schema name to use to make annotations. Only needed if the table_name is not set, by default None</p> <code>None</code> <code>update</code> <code>bool</code> <p>Set to True if individual annotations are going to be updated, by default False.</p> <code>False</code> <code>id_field</code> <code>bool</code> <p>Set to True if id fields are to be specified. Not needed if update is True, which always needs id fields. Optional, by default False</p> <code>False</code> <code>table_resolution</code> <code>list - like or None</code> <p>Voxel resolution of spatial points in the table in nanometers. This is found automatically from the info service if a table name is provided, by default None. If annotation_resolution is also set, this allows points to be scaled correctly for the table.</p> <code>None</code> <code>annotation_resolution</code> <code>list - like</code> <p>Voxel resolution of spatial points provided by the user when creating annotations. If the table resolution is also available (manually or from the info service), annotations are correctly rescaled for the volume. By default, None.</p> <code>None</code>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.update_annotation","title":"<code>update_annotation(table_name, data, aligned_volume_name=None)</code>","text":"<p>Update one or more new annotations to a table in the AnnotationEngine. Updating is implemented by invalidating the old annotation and inserting a new annotation row, which will receive a new primary key ID.</p> Notes <p>If annotations ids were user provided upon insertion the database will autoincrement from the current max id in the table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>data</code> <code>(dict or list)</code> <p>A list of (or a single) dict of schematized annotation data matching the target table. each dict must contain an \"id\" field which is the ID of the annotation to update</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON: a list of new annotation IDs.</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.update_annotation_df","title":"<code>update_annotation_df(table_name, df, position_columns, aligned_volume_name=None)</code>","text":"<p>Update one or more annotations to a table in the AnnotationEngine using a dataframe as format. Updating is implemented by invalidating the old annotation and inserting a new annotation row, which will receive a new primary key ID.</p> Notes <p>If annotations ids were user provided upon insertion the database will autoincrement from the current max id in the table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Name of the table where annotations will be added</p> required <code>df</code> <code>DataFrame</code> <p>A pandas dataframe containing the annotations. Columns should be fields in schema, position columns need to be called out in position_columns argument.</p> required <code>position_columns</code> <code>Iterable[str] or Mapping[str, str] or None</code> <p>if None, will look for all columns with 'X_position' in the name and assume they go in fields called \"X\". if Iterable assumes each column given ends in _position. (i.e. ['pt_position'] if 'pt' is the name of the position field in schema) if Mapping, keys are names of columns in dataframe, values are the names of the fields (i.e. {'pt_column': 'pt'} would be correct if you had one column named 'pt_column' which needed to go into a schema with a position column called 'pt')</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>json</code> <p>Response JSON</p>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.update_metadata","title":"<code>update_metadata(table_name, description=None, flat_segmentation_source=None, read_permission=None, write_permission=None, user_id=None, notice_text=None, aligned_volume_name=None)</code>","text":"<p>Update the metadata on an existing table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> required <code>description</code> <code>str</code> <p>Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>flat_segmentation_source</code> <code>str</code> <p>Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>read_permission</code> <code>str</code> <p>What permissions to give the table for reading. One of PRIVATE: only you can read this table. Intended to be used for sorting out bugs. GROUP: only members that share a group with you can read (intended for within group vetting) PUBLIC: anyone with permissions to read this datastack can read this data Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>write_permission</code> <code>str</code> <p>What permissions to give the table for writing.  One of PRIVATE: only you can write to this table GROUP: only members that share a group with you can write (excluding some groups) PUBLIC: Anyone can write to this table. Note all data is logged, and deletes are done by marking rows as deleted, so all data is always recoverable Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>user_id</code> <code>int</code> <p>Note, if you use this you will not be able to update the metadata on this table any longer and depending on permissions may not be able to read or write to it Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>notice_text</code> <code>str</code> <p>Text the user will see when querying this table. Can be used to warn users of flaws, and uncertainty in the data, or to advertise citations that should be used with this table. If you wish to remove the notice_text pass an empty string, or \"none\", \"None\", \"NONE\", or any other capitaliztion of the word \"none\". Defaults to the None type, in which case no change will be made to this metadata field.</p> <code>None</code> <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code>"},{"location":"api/annotation/#caveclient.annotationengine.AnnotationClient.upload_staged_annotations","title":"<code>upload_staged_annotations(staged_annos, aligned_volume_name=None)</code>","text":"<p>Upload annotations directly from an Annotation Guide object. This method uses the options specified in the object, including table name and if the annotation is an update or not.</p> <p>Parameters:</p> Name Type Description Default <code>staged_annos</code> <code>AnnotationGuide</code> <p>AnnotationGuide object with a specified table name and a collection of annotations already filled in.</p> required <code>aligned_volume_name</code> <code>str or None</code> <p>Name of the aligned_volume. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>List or dict</code> <p>If new annotations are posted, a list of ids. If annotations are being updated, a dictionary with the mapping from old ids to new ids.</p>"},{"location":"api/auth/","title":"client.auth","text":"<p>Client to find and use auth tokens to access the  other services.</p> <p>Methods:</p> Name Description <code>get_group_users</code> <p>Get users in a group</p> <code>get_new_token</code> <p>Currently, returns instructions for getting a new token based on the current settings and saving it to the local environment. New OAuth tokens are currently not able to be retrieved programmatically.</p> <code>get_token</code> <p>Load a token with a given key the specified token file</p> <code>get_tokens</code> <p>Get the tokens setup for this users</p> <code>get_user_information</code> <p>Get user data.</p> <code>save_token</code> <p>Conveniently save a token in the correct format.</p> <code>setup_token</code> <p>Currently, returns instructions for getting your auth token based on the current settings and saving it to the local environment.</p> <p>Attributes:</p> Name Type Description <code>request_header</code> <p>Formatted request header with the specified token</p> <code>token</code> <p>Secret token used to authenticate yourself to the Connectome Annotation Versioning Engine services.</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.request_header","title":"<code>request_header</code>  <code>property</code>","text":"<p>Formatted request header with the specified token</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.token","title":"<code>token</code>  <code>property</code> <code>writable</code>","text":"<p>Secret token used to authenticate yourself to the Connectome Annotation Versioning Engine services.</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_group_users","title":"<code>get_group_users(group_id)</code>","text":"<p>Get users in a group</p> <p>Parameters:</p> Name Type Description Default <code>group_id</code> <code>int</code> <p>ID value for a given group</p> required <p>Returns:</p> Type Description <code>list</code> <p>List of dicts of user ids. Returns empty list if group does not exist.</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_new_token","title":"<code>get_new_token(open=False, no_text=False)</code>","text":"<p>Currently, returns instructions for getting a new token based on the current settings and saving it to the local environment. New OAuth tokens are currently not able to be retrieved programmatically.</p> <p>Parameters:</p> Name Type Description Default <code>open</code> <code>bool</code> <p>If True, opens a web browser to the web page where you can generate a new token.</p> <code>False</code>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_token","title":"<code>get_token(token_key=None)</code>","text":"<p>Load a token with a given key the specified token file</p> <p>Parameters:</p> Name Type Description Default <code>token_key</code> <code>str or None</code> <p>key in the token file JSON, by default None. If None, uses 'token'.</p> <code>None</code>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_tokens","title":"<code>get_tokens()</code>","text":"<p>Get the tokens setup for this users</p> <p>Returns:</p> Type Description <code>list[dict]:</code> <p>a list of dictionary of tokens, each with the keys \"id\": the id of this token \"token\": the token (str) \"user_id\": the users id (should be your ID)</p>"},{"location":"api/auth/#caveclient.auth.AuthClient.get_user_information","title":"<code>get_user_information(user_ids)</code>","text":"<p>Get user data.</p> <p>Parameters:</p> Name Type Description Default <code>user_ids</code> <code>list of int</code> <p>user_ids to look up</p> required"},{"location":"api/auth/#caveclient.auth.AuthClient.save_token","title":"<code>save_token(token=None, token_key=default_token_key, overwrite=False, token_file=None, switch_token=True, write_to_server_file=True, ignore_readonly=True, local_server=True)</code>","text":"<p>Conveniently save a token in the correct format.</p> <p>After getting a new token by following the instructions in <code>authclient.get_new_token()</code>, you can save it with a fully default configuration by running:</p> <p>token = 'my_shiny_new_token'</p> <p>authclient.save_token(token=token)</p> <p>Now on next load, authclient=AuthClient() will make an authclient instance using this token. If you would like to specify more information about the json file where the token will be stored, see the parameters below.</p> <p>Parameters:</p> Name Type Description Default <code>token</code> <code>str</code> <p>New token to save, by default None</p> <code>None</code> <code>token_key</code> <code>str</code> <p>Key for the token in the token_file json, by default \"token\"</p> <code>default_token_key</code> <code>overwrite</code> <code>bool</code> <p>Allow an existing token to be changed, by default False</p> <code>False</code> <code>token_file</code> <code>str</code> <p>Path to the token file, by default None. If None, uses the default file location specified above.</p> <code>None</code> <code>switch_token</code> <code>bool</code> <p>If True, switch the auth client over into using the new token, by default True</p> <code>True</code> <code>write_to_server_file</code> <code>bool</code> <p>If True, will write token to a server specific file to support this machine interacting with multiple auth servers.</p> <code>True</code> <code>ignore_readonly</code> <code>bool</code> <p>If True, will only attempt to save a token if the directory is writeable.</p> <code>True</code> <code>local_server</code> <code>bool</code> <p>If True, saves the token to the local server file as well.</p> <code>True</code>"},{"location":"api/auth/#caveclient.auth.AuthClient.setup_token","title":"<code>setup_token(make_new=True, open=True)</code>","text":"<p>Currently, returns instructions for getting your auth token based on the current settings and saving it to the local environment. New OAuth tokens are currently not able to be retrieved programmatically.</p> <p>Parameters:</p> Name Type Description Default <code>make_new</code> <code>bool</code> <p>If True, will make a new token, else prompt you to open a page to retrieve an existing token.</p> <code>True</code> <code>open</code> <code>bool</code> <p>If True, opens a web browser to the web page where you can retrieve a token.</p> <code>True</code>"},{"location":"api/chunkedgraph/","title":"client.chunkedgraph","text":"<p>Client for interacting with the chunkedgraph.</p> <p>Methods:</p> Name Description <code>get_roots</code> <p>Get the root ID for a list of supervoxels.</p> <code>get_root_id</code> <p>Get the root ID for a specified supervoxel.</p> <code>get_minimal_covering_nodes</code> <p>Get the minimal covering nodes for a list of root IDs.</p> <code>get_merge_log</code> <p>Get the merge log (splits and merges) for an object.</p> <code>get_change_log</code> <p>Get the change log (splits and merges) for an object.</p> <code>get_user_operations</code> <p>Get operation details for a user ID. Currently, this is only available to</p> <code>get_tabular_change_log</code> <p>Get a detailed changelog for neurons.</p> <code>get_leaves_many</code> <p>Get all supervoxels for a list of root IDs.</p> <code>get_leaves</code> <p>Get all supervoxels for a root ID.</p> <code>do_merge</code> <p>Perform a merge on the chunked graph.</p> <code>undo_operation</code> <p>Undo an operation.</p> <code>execute_split</code> <p>Execute a multicut split based on points or supervoxels.</p> <code>preview_split</code> <p>Get supervoxel connected components from a preview multicut split.</p> <code>get_children</code> <p>Get the children of a node in the chunked graph hierarchy.</p> <code>get_contact_sites</code> <p>Get contacts for a root ID.</p> <code>find_path</code> <p>Find a path between two locations on a root ID using the level 2 chunked</p> <code>get_subgraph</code> <p>Get subgraph of root id within a bounding box.</p> <code>level2_chunk_graph</code> <p>Get graph of level 2 chunks, the smallest agglomeration level above supervoxels.</p> <code>remesh_level2_chunks</code> <p>Submit specific level 2 chunks to be remeshed in case of a problem.</p> <code>get_operation_details</code> <p>Get the details of a list of operations.</p> <code>get_lineage_graph</code> <p>Returns the lineage graph for a root ID, optionally cut off in the past or</p> <code>get_latest_roots</code> <p>Returns root IDs that are related to the given <code>root_id</code> at a given</p> <code>get_original_roots</code> <p>Returns root IDs that are the latest successors of a given root ID.</p> <code>is_latest_roots</code> <p>Check whether these root IDs are still a root at this timestamp.</p> <code>suggest_latest_roots</code> <p>Suggest latest roots for a given root id, based on overlap of component</p> <code>is_valid_nodes</code> <p>Check whether nodes are valid for given timestamp range.</p> <code>get_root_timestamps</code> <p>Retrieves timestamps when roots where created.</p> <code>get_past_ids</code> <p>For a set of root IDs, get the list of IDs at a past or future time point</p> <code>get_delta_roots</code> <p>Get the list of roots that have changed between <code>timetamp_past</code> and</p> <code>get_oldest_timestamp</code> <p>Get the oldest timestamp in the database.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p> <code>timestamp</code> <code>Optional[datetime]</code> <p>The default timestamp for queries which expect a timestamp. If None, uses the</p> <code>segmentation_info</code> <p>Complete segmentation metadata</p> <code>base_resolution</code> <p>MIP 0 resolution for voxels assumed by the ChunkedGraph</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.timestamp","title":"<code>timestamp: Optional[datetime.datetime]</code>  <code>property</code> <code>writable</code>","text":"<p>The default timestamp for queries which expect a timestamp. If None, uses the current time.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.segmentation_info","title":"<code>segmentation_info</code>  <code>property</code>","text":"<p>Complete segmentation metadata</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.base_resolution","title":"<code>base_resolution</code>  <code>property</code>","text":"<p>MIP 0 resolution for voxels assumed by the ChunkedGraph</p> <p>Returns:</p> Type Description <code>list</code> <p>3-long list of x/y/z voxel dimensions in nm</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_roots","title":"<code>get_roots(supervoxel_ids, timestamp=None, stop_layer=None)</code>","text":"<p>Get the root ID for a list of supervoxels.</p> <p>Parameters:</p> Name Type Description Default <code>supervoxel_ids</code> <code>list or np.array of int</code> <p>Supervoxel IDs to look up.</p> required <code>timestamp</code> <code>datetime</code> <p>UTC datetime to specify the state of the chunkedgraph at which to query, by default None. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <code>stop_layer</code> <code>int or None</code> <p>If True, looks up IDs only up to a given stop layer. Default is None.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of np.uint64</code> <p>Root IDs containing each supervoxel.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_root_id","title":"<code>get_root_id(supervoxel_id, timestamp=None, level2=False)</code>","text":"<p>Get the root ID for a specified supervoxel.</p> <p>Parameters:</p> Name Type Description Default <code>supervoxel_id</code> <code>int</code> <p>Supervoxel id value</p> required <code>timestamp</code> <code>datetime</code> <p>UTC datetime to specify the state of the chunkedgraph at which to query, by default None. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>int64</code> <p>Root ID containing the supervoxel.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_minimal_covering_nodes","title":"<code>get_minimal_covering_nodes(node_ids)</code>","text":"<p>Get the minimal covering nodes for a list of root IDs.</p> <p>Parameters:</p> Name Type Description Default <code>nodes_ids</code> <code>Iterable of int or np.int64</code> <p>List of root IDs to query.</p> required <p>Returns:</p> Type Description <code>np.array of np.int64:</code> <p>List of PCG node_ids that minimally and exactly cover the input nodes</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_merge_log","title":"<code>get_merge_log(root_id)</code>","text":"<p>Get the merge log (splits and merges) for an object.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID to look up.</p> required <p>Returns:</p> Type Description <code>list</code> <p>List of merge events in the history of the object.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_change_log","title":"<code>get_change_log(root_id, filtered=True)</code>","text":"<p>Get the change log (splits and merges) for an object.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID to look up.</p> required <code>filtered</code> <code>bool</code> <p>Whether to filter the change log to only include splits and merges which affect the final state of the object (<code>filtered=True</code>), as opposed to including edit history for objects which as some point were split from the query object <code>root_id</code> (<code>filtered=False</code>). Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary summarizing split and merge events in the object history, containing the following keys:</p> <ul> <li>\"n_merges\": int<ul> <li>Number of merges</li> </ul> </li> <li>\"n_splits\": int<ul> <li>Number of splits</li> </ul> </li> <li>\"operations_ids\": list of int<ul> <li>Identifiers for each operation</li> </ul> </li> <li>\"past_ids\": list of int<ul> <li>Previous root ids for this object</li> </ul> </li> <li>\"user_info\": dict of dict<ul> <li>Dictionary keyed by user (string) to a dictionary specifying how many   merges and splits that user performed on this object</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_user_operations","title":"<code>get_user_operations(user_id, timestamp_start, include_undo=True, timestamp_end=None)</code>","text":"<p>Get operation details for a user ID. Currently, this is only available to admins.</p> <p>Parameters:</p> Name Type Description Default <code>user_id</code> <code>int</code> <p>User ID to query (use 0 for all users (admin only)).</p> required <code>timestamp_start</code> <code>datetime</code> <p>Timestamp to start filter (UTC).</p> required <code>include_undo</code> <code>bool</code> <p>Whether to include undos. Defaults to True.</p> <code>True</code> <code>timestamp_end</code> <code>datetime</code> <p>Timestamp to end filter (UTC). Defaults to now.</p> <code>None</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>DataFrame including the following columns:</p> <ul> <li>\"operation_id\": int<ul> <li>Identifier for the operation.</li> </ul> </li> <li>\"timestamp\": datetime.datetime<ul> <li>Timestamp of the operation.</li> </ul> </li> <li>\"user_id\": int<ul> <li>User who performed the operation.</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_tabular_change_log","title":"<code>get_tabular_change_log(root_ids, filtered=True)</code>","text":"<p>Get a detailed changelog for neurons.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>list of int</code> <p>Object root IDs to look up.</p> required <code>filtered</code> <code>bool</code> <p>Whether to filter the change log to only include splits and merges which affect the final state of the object (<code>filtered=True</code>), as opposed to including edit history for objects which as some point were split from the query objects in <code>root_ids</code> (<code>filtered=False</code>). Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict of pd.DataFrame</code> <p>The keys are the root IDs, and the values are DataFrames with the following columns and datatypes:</p> <ul> <li>\"operation_id\": int<ul> <li>Identifier for the operation.</li> </ul> </li> <li>\"timestamp\": int<ul> <li>Timestamp of the operation, provided in milliseconds. To convert to datetime, use <code>datetime.datetime.utcfromtimestamp(timestamp/1000)</code>.</li> </ul> </li> <li>\"user_id\": int<ul> <li>User who performed the operation.</li> </ul> </li> <li>\"before_root_ids: list of int<ul> <li>Root IDs of objects that existed before the operation.</li> </ul> </li> <li>\"after_root_ids: list of int<ul> <li>Root IDs of objects created by the operation. Note that this only records the root id that was kept as part of the query object, so there will only be one in this list.</li> </ul> </li> <li>\"is_merge\": bool<ul> <li>Whether the operation was a merge.</li> </ul> </li> <li>\"user_name\": str<ul> <li>Name of the user who performed the operation.</li> </ul> </li> <li>\"user_affiliation\": str<ul> <li>Affiliation of the user who performed the operation.</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_leaves_many","title":"<code>get_leaves_many(root_ids, bounds=None, stop_layer=None)</code>","text":"<p>Get all supervoxels for a list of root IDs.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>Iterable</code> <p>Root IDs to query.</p> required <code>bounds</code> <p>If specified, returns supervoxels within a 3x2 numpy array of bounds <code>[[minx,maxx],[miny,maxy],[minz,maxz]]</code>. If None, finds all supervoxels.</p> <code>None</code> <code>stop_layer</code> <code>int</code> <p>If specified, returns chunkedgraph nodes at layer <code>stop_layer</code> default will be <code>stop_layer=1</code> (supervoxels).</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dict relating ids to contacts</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_leaves","title":"<code>get_leaves(root_id, bounds=None, stop_layer=None)</code>","text":"<p>Get all supervoxels for a root ID.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID to query.</p> required <code>bounds</code> <p>If specified, returns supervoxels within a 3x2 numpy array of bounds <code>[[minx,maxx],[miny,maxy],[minz,maxz]]</code>. If None, finds all supervoxels.</p> <code>None</code> <code>stop_layer</code> <code>int</code> <p>If specified, returns chunkedgraph nodes at layer <code>stop_layer</code> default will be <code>stop_layer=1</code> (supervoxels).</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of np.int64</code> <p>Array of supervoxel IDs (or node ids if <code>stop_layer&gt;1</code>).</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.do_merge","title":"<code>do_merge(supervoxels, coords, resolution=(4, 4, 40))</code>","text":"<p>Perform a merge on the chunked graph.</p> <p>Parameters:</p> Name Type Description Default <code>supervoxels</code> <code>iterable</code> <p>An N-long list of supervoxels to merge.</p> required <code>coords</code> <code>array</code> <p>An Nx3 array of coordinates of the supervoxels in units of <code>resolution</code>.</p> required <code>resolution</code> <code>tuple</code> <p>What to multiply <code>coords</code> by to get nanometers. Defaults to (4,4,40).</p> <code>(4, 4, 40)</code>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.undo_operation","title":"<code>undo_operation(operation_id)</code>","text":"<p>Undo an operation.</p> <p>Parameters:</p> Name Type Description Default <code>operation_id</code> <code>int</code> <p>Operation ID to undo.</p> required <p>Returns:</p> Type Description <code>dict</code>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.execute_split","title":"<code>execute_split(source_points, sink_points, root_id, source_supervoxels=None, sink_supervoxels=None)</code>","text":"<p>Execute a multicut split based on points or supervoxels.</p> <p>Parameters:</p> Name Type Description Default <code>source_points</code> <code>array or list</code> <p>Nx3 list or array of 3d points in nm coordinates for source points (red).</p> required <code>sink_points</code> <code>array or list</code> <p>Mx3 list or array of 3d points in nm coordinates for sink points (blue).</p> required <code>root_id</code> <code>int</code> <p>Root ID of object to do split preview.</p> required <code>source_supervoxels</code> <code>(array, list or None)</code> <p>If providing source supervoxels, an N-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <code>sink_supervoxels</code> <code>(array, list or None)</code> <p>If providing sink supervoxels, an M-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <p>Returns:</p> Name Type Description <code>operation_id</code> <code>int</code> <p>Unique ID of the split operation</p> <code>new_root_ids</code> <code>list of int</code> <p>List of new root IDs resulting from the split operation.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.preview_split","title":"<code>preview_split(source_points, sink_points, root_id, source_supervoxels=None, sink_supervoxels=None, return_additional_ccs=False)</code>","text":"<p>Get supervoxel connected components from a preview multicut split.</p> <p>Parameters:</p> Name Type Description Default <code>source_points</code> <code>array or list</code> <p>Nx3 list or array of 3d points in nm coordinates for source points (red).</p> required <code>sink_points</code> <code>array or list</code> <p>Mx3 list or array of 3d points in nm coordinates for sink points (blue).</p> required <code>root_id</code> <code>int</code> <p>Root ID of object to do split preview.</p> required <code>source_supervoxels</code> <code>(array, list or None)</code> <p>If providing source supervoxels, an N-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <code>sink_supervoxels</code> <code>(array, list or None)</code> <p>If providing sink supervoxels, an M-length array of supervoxel IDs or Nones matched to source points. If None, treats as a full array of Nones. By default None.</p> <code>None</code> <code>return_additional_ccs</code> <code>bool</code> <p>If True, returns any additional connected components beyond the ones with source and sink points. In most situations, this can be ignored. By default, False.</p> <code>False</code> <p>Returns:</p> Name Type Description <code>source_connected_component</code> <code>list</code> <p>Supervoxel IDs in the component with the most source points.</p> <code>sink_connected_component</code> <code>list</code> <p>Supervoxel IDs in the component with the most sink points.</p> <code>successful_split</code> <code>bool</code> <p>True if the split worked.</p> <code>other_connected_components (optional) : list of lists of int</code> <p>List of lists of supervoxel IDs for any other resulting connected components. Only returned if <code>return_additional_ccs</code> is True.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_children","title":"<code>get_children(node_id)</code>","text":"<p>Get the children of a node in the chunked graph hierarchy.</p> <p>Parameters:</p> Name Type Description Default <code>node_id</code> <code>int</code> <p>Node ID to query.</p> required <p>Returns:</p> Type Description <code>np.array of np.int64</code> <p>IDs of child nodes.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_contact_sites","title":"<code>get_contact_sites(root_id, bounds, calc_partners=False)</code>","text":"<p>Get contacts for a root ID.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID to query.</p> required <code>bounds</code> <p>Bounds within a 3x2 numpy array of bounds <code>[[minx,maxx],[miny,maxy],[minz,maxz]]</code> for which to find contacts. Running this query without bounds is too slow.</p> required <code>calc_partners</code> <code>bool</code> <p>If True, get partner root IDs. By default, False.</p> <code>False</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dict relating ids to contacts</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.find_path","title":"<code>find_path(root_id, src_pt, dst_pt, precision_mode=False)</code>","text":"<p>Find a path between two locations on a root ID using the level 2 chunked graph.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID to query.</p> required <code>src_pt</code> <code>array</code> <p>3-element array of xyz coordinates in nm for the source point.</p> required <code>dst_pt</code> <code>array</code> <p>3-element array of xyz coordinates in nm for the destination point.</p> required <code>precision_mode</code> <code>bool</code> <p>Whether to perform the search in precision mode. Defaults to False.</p> <code>False</code> <p>Returns:</p> Name Type Description <code>centroids_list</code> <code>array</code> <p>Array of centroids along the path.</p> <code>l2_path</code> <code>np.array of int</code> <p>Array of level 2 chunk IDs along the path.</p> <code>failed_l2_ids</code> <code>np.array of int</code> <p>Array of level 2 chunk IDs that failed to find a path.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_subgraph","title":"<code>get_subgraph(root_id, bounds)</code>","text":"<p>Get subgraph of root id within a bounding box.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root (or any node ID) of chunked graph to query.</p> required <code>bounds</code> <code>array</code> <p>3x2 bounding box (x,y,z) x (min,max) in chunked graph coordinates.</p> required <p>Returns:</p> Type Description <code>np.array of np.int64</code> <p>Node IDs in the subgraph.</p> <code>np.array of np.double</code> <p>Affinities of edges in the subgraph.</p> <code>np.array of np.int32</code> <p>Areas of nodes in the subgraph.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.level2_chunk_graph","title":"<code>level2_chunk_graph(root_id, bounds=None)</code>","text":"<p>Get graph of level 2 chunks, the smallest agglomeration level above supervoxels.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root id of object</p> required <code>bounds</code> <code>array</code> <p>3x2 bounding box (x,y,z) x (min,max) in chunked graph coordinates (use <code>client.chunkedgraph.base_resolution</code> to view this default resolution for your chunkedgraph client). Note that the result will include any level 2 nodes which have chunk boundaries within some part of this bounding box, meaning that the representative point for a given level 2 node could still be slightly outside of these bounds. If None, returns all level 2 chunks for the root ID.</p> <code>None</code> <p>Returns:</p> Type Description <code>list of list</code> <p>Edge list for level 2 chunked graph. Each element of the list is an edge, and each edge is a list of two node IDs (source and target).</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.remesh_level2_chunks","title":"<code>remesh_level2_chunks(chunk_ids)</code>","text":"<p>Submit specific level 2 chunks to be remeshed in case of a problem.</p> <p>Parameters:</p> Name Type Description Default <code>chunk_ids</code> <code>list</code> <p>List of level 2 chunk IDs.</p> required"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_operation_details","title":"<code>get_operation_details(operation_ids)</code>","text":"<p>Get the details of a list of operations.</p> <p>Parameters:</p> Name Type Description Default <code>operation_ids</code> <code>Iterable[int]</code> <p>List/array of operation IDs.</p> required <p>Returns:</p> Type Description <code>dict of str to dict</code> <p>A dict of dicts of operation info, keys are operation IDs (as strings), values are a dictionary of operation info for the operation. These dictionaries contain the following keys:</p> <ul> <li>\"added_edges\"/\"removed_edges\": list of list of int<ul> <li>List of edges added (if a merge) or removed (if a split) by this operation. Each edge is a list of two supervoxel IDs (source and target).</li> </ul> </li> <li>\"roots\": list of int<ul> <li>List of root IDs that were created by this operation.</li> </ul> </li> <li>\"sink_coords\": list of list of int<ul> <li>List of sink coordinates for this operation. The sink is one of the points placed by the user when specifying the operation. Each sink coordinate is a list of three integers (x, y, z), corresponding to spatial coordinates in segmentation voxel space.</li> </ul> </li> <li>\"source_coords\": list of list of int<ul> <li>List of source coordinates for this operation. The source is one of the points placed by the user when specifying the operation. Each source coordinate is a list of three integers (x, y, z), corresponding to spatial coordinates in segmentation voxel space.</li> </ul> </li> <li>\"timestamp\": str<ul> <li>Timestamp of the operation.</li> </ul> </li> <li>\"user\": str<ul> <li>User ID number who performed the operation (as a string).</li> </ul> </li> </ul>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_lineage_graph","title":"<code>get_lineage_graph(root_id, timestamp_past=None, timestamp_future=None, as_nx_graph=False, exclude_links_to_future=False, exclude_links_to_past=False)</code>","text":"<p>Returns the lineage graph for a root ID, optionally cut off in the past or the future.</p> <p>Each change in the chunked graph creates a new root ID for the object after that change. This function returns a graph of all root IDs for a given object, tracing the history of the object in terms of merges and splits.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID.</p> required <code>timestamp_past</code> <code>datetime or None</code> <p>Cutoff for the lineage graph backwards in time. By default, None.</p> <code>None</code> <code>timestamp_future</code> <code>datetime or None</code> <p>Cutoff for the lineage graph going forwards in time. By default, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <code>as_nx_graph</code> <p>If True, a NetworkX graph is returned.</p> <code>False</code> <code>exclude_links_to_future</code> <p>If True, links from nodes before <code>timestamp_future</code> to after <code>timestamp_future</code> are removed. If False, the link(s) which has one node before timestamp and one node after timestamp is kept.</p> <code>False</code> <code>exclude_links_to_past</code> <p>If True, links from nodes before <code>timestamp_past</code> to after <code>timestamp_past</code> are removed. If False, the link(s) which has one node before timestamp and one node after timestamp is kept.</p> <code>False</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary describing the lineage graph and operations for the root ID. Not returned if <code>as_nx_graph</code> is True. The dictionary contains the following keys:</p> <ul> <li>\"directed\" : bool<ul> <li>Whether the graph is directed.</li> </ul> </li> <li>\"graph\" : dict<ul> <li>Dictionary of graph attributes.</li> </ul> </li> <li>\"links\" : list of dict<ul> <li>Each element of the list is a dictionary describing an edge in the lineage graph as \"source\" and \"target\" keys.</li> </ul> </li> <li>\"multigraph\" : bool<ul> <li>Whether the graph is a multigraph.</li> </ul> </li> <li>\"nodes\" : list of dict<ul> <li>Each element of the list is a dictionary describing a node in the lineage graph, usually with \"id\", \"timestamp\", and \"operation_id\" keys.</li> </ul> </li> </ul> <code>DiGraph</code> <p>NetworkX directed graph of the lineage graph. Only returned if <code>as_nx_graph</code> is True.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_latest_roots","title":"<code>get_latest_roots(root_id, timestamp=None, timestamp_future=None)</code>","text":"<p>Returns root IDs that are related to the given <code>root_id</code> at a given timestamp. Can be used to find the \"latest\" root IDs associated with an object.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID.</p> required <code>timestamp</code> <code>datetime or None</code> <p>Timestamp of where to query IDs from. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <code>timestamp_future</code> <code>datetime or None</code> <p>DEPRECATED name, use <code>timestamp</code> instead. Timestamp to suggest IDs from (note can be in the past relative to the root). By default, None.</p> <code>None</code> <p>Returns:</p> Type Description <code>ndarray</code> <p>1d array with all latest successors.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_original_roots","title":"<code>get_original_roots(root_id, timestamp_past=None)</code>","text":"<p>Returns root IDs that are the latest successors of a given root ID.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Object root ID.</p> required <code>timestamp_past</code> <code>datetime or None</code> <p>Cutoff for the search going backwards in time. By default, None.</p> <code>None</code> <p>Returns:</p> Type Description <code>ndarray</code> <p>1d array with all latest successors.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.is_latest_roots","title":"<code>is_latest_roots(root_ids, timestamp=None)</code>","text":"<p>Check whether these root IDs are still a root at this timestamp.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>array-like of int</code> <p>Root IDs to check.</p> required <code>timestamp</code> <code>datetime</code> <p>Timestamp to check whether these IDs are valid root IDs in the chunked graph. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of bool</code> <p>Array of whether these are valid root IDs.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.suggest_latest_roots","title":"<code>suggest_latest_roots(root_id, timestamp=None, stop_layer=None, return_all=False, return_fraction_overlap=False)</code>","text":"<p>Suggest latest roots for a given root id, based on overlap of component chunk IDs. Note that edits change chunk IDs, and so this effectively measures the fraction of unchanged chunks at a given chunk layer, which sets the size scale of chunks. Higher layers are coarser.</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>Root ID of the potentially outdated object.</p> required <code>timestamp</code> <code>datetime</code> <p>Datetime at which \"latest\" roots are being computed, by default None. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time. Note that this has to be a timestamp after the creation of the <code>root_id</code>.</p> <code>None</code> <code>stop_layer</code> <code>int</code> <p>Chunk level at which to compute overlap, by default None. No value will take the 4th from the top layer, which emphasizes speed and works well for larger objects. Lower values are slower but more fine-grained. Values under 2 (i.e. supervoxels) are not recommended except in extremely fine grained scenarios.</p> <code>None</code> <code>return_all</code> <code>bool</code> <p>If True, return all current IDs sorted from most overlap to least, by default False. If False, only the top is returned.</p> <code>False</code> <code>return_fraction_overlap</code> <code>bool</code> <p>If True, return all fractions sorted by most overlap to least, by default False. If False, only the top value is returned.</p> <code>False</code>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.is_valid_nodes","title":"<code>is_valid_nodes(node_ids, start_timestamp=None, end_timestamp=None)</code>","text":"<p>Check whether nodes are valid for given timestamp range.</p> <p>Valid is defined as existing in the chunked graph. This makes no statement about these IDs being roots, supervoxel or anything in-between. It also does not take into account whether a root ID has since been edited.</p> <p>Parameters:</p> Name Type Description Default <code>node_ids</code> <code>array-like of int</code> <p>Node IDs to check.</p> required <code>start_timestamp</code> <code>datetime</code> <p>Timestamp to check whether these IDs were valid after this timestamp. Defaults to None (assumes now).</p> <code>None</code> <code>end_timestamp</code> <code>datetime</code> <p>Timestamp to check whether these IDs were valid before this timestamp. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of bool</code> <p>Array of whether these are valid IDs.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_root_timestamps","title":"<code>get_root_timestamps(root_ids, latest=False, timestamp=None)</code>","text":"<p>Retrieves timestamps when roots where created.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <p>Iterable of root IDs to query.</p> required <code>latest</code> <code>bool</code> <p>If False,  returns the first timestamp that the root_id was valid for each root ID. If True, returns the newest/latest timestamp for each root ID. Note, this will return the timestamp at which the query was run when the root is currently valid. This means that you will get a different answer if you make this same query at a later time if you don't specify a timestamp parameter.</p> <code>False</code> <code>timestamp</code> <code>datetime</code> <p>Timestamp to query when using latest=True. Use this to provide consistent results for a particular timestamp. If an ID is still valid at a point in the future past this timestamp, the query will still return this timestamp as the latest moment in time. An error will occur if you provide a timestamp for which the root ID is not valid. If None, uses the <code>timestamp</code> property for this client, which defaults to the current time.</p> <code>None</code> <p>Returns:</p> Type Description <code>np.array of datetime.datetime</code> <p>Array of timestamps when <code>root_ids</code> were created.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_past_ids","title":"<code>get_past_ids(root_ids, timestamp_past=None, timestamp_future=None)</code>","text":"<p>For a set of root IDs, get the list of IDs at a past or future time point that could contain parts of the same object.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>Iterable of int</code> <p>Iterable of root IDs to query.</p> required <code>timestamp_past</code> <code>datetime or None</code> <p>Time of a point in the past for which to look up root ids. Default is None.</p> <code>None</code> <code>timestamp_future</code> <code>datetime or None</code> <p>Time of a point in the future for which to look up root ids. Not implemented on the server currently. Default is None.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dict with keys \"future_id_map\" and \"past_id_map\". Each is a dict whose keys are the supplied <code>root_ids</code> and whose values are the list of related root IDs at <code>timestamp_past</code>/<code>timestamp_future</code>.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_delta_roots","title":"<code>get_delta_roots(timestamp_past, timestamp_future=datetime.datetime.now(datetime.timezone.utc))</code>","text":"<p>Get the list of roots that have changed between <code>timetamp_past</code> and <code>timestamp_future</code>.</p> <p>Parameters:</p> Name Type Description Default <code>timestamp_past</code> <code>datetime</code> <p>Past timepoint to query</p> required <code>timestamp_future</code> <code>datetime</code> <p>Future timepoint to query. Defaults to <code>datetime.datetime.now(datetime.timezone.utc)</code>.</p> <code>now(utc)</code> <p>Returns:</p> Name Type Description <code>old_roots</code> <code>np.ndarray of np.int64</code> <p>Roots that have expired in that interval.</p> <code>new_roots</code> <code>np.ndarray of np.int64</code> <p>Roots that are new in that interval.</p>"},{"location":"api/chunkedgraph/#caveclient.chunkedgraph.ChunkedGraphClient.get_oldest_timestamp","title":"<code>get_oldest_timestamp()</code>","text":"<p>Get the oldest timestamp in the database.</p> <p>Returns:</p> Type Description <code>datetime</code> <p>Oldest timestamp in the database.</p>"},{"location":"api/client/","title":"caveclient.CAVEclient","text":"<p>A manager for all clients sharing common datastack and authentication information.</p> <p>This client wraps all the other clients and keeps track of the things that need to be consistent across them. To instantiate a client:</p> <pre><code>from caveclient import CAVEclient\n\nclient = CAVEclient(datastack_name='my_datastack',\n                    server_address='www.myserver.com',\n                    auth_token_file='~/.mysecrets/secrets.json')\n</code></pre> <p>Then</p> <ul> <li><code>client.annotation</code> is an <code>AnnotationClient</code> (see client.annotation)</li> <li><code>client.auth</code> is an <code>AuthClient</code> (see client.auth)</li> <li><code>client.chunkedgraph</code> is a <code>ChunkedGraphClient</code> (see client.chunkedgraph)</li> <li><code>client.info</code> is an <code>InfoServiceClient</code> (see client.info)</li> <li><code>client.l2cache</code> is an <code>L2CacheClient</code> (see client.l2cache)</li> <li><code>client.materialize</code> is a <code>MaterializationClient</code> (see client.materialize)</li> <li><code>client.skeleton</code> is a <code>SkeletonClient</code> (see client.skeleton)</li> <li><code>client.schema</code> is a <code>SchemaClient</code> (see client.schema)</li> <li><code>client.state</code> is a neuroglancer <code>JSONService</code> (see client.state)</li> </ul> <p>All subclients are loaded lazily and share the same datastack name, server address, and auth tokens where used.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>Datastack name for the services. Almost all services need this and will not work if it is not passed.</p> <code>None</code> <code>server_address</code> <code>str or None</code> <p>URL of the framework server. If None, chooses the default server global.daf-apis.com. Optional, defaults to None.</p> <code>None</code> <code>auth_token_file</code> <code>str or None</code> <p>Path to a json file containing the auth token. If None, uses the default location. See Auth client documentation. Optional, defaults to None.</p> <code>default_token_file</code> <code>auth_token_key</code> <code>str</code> <p>Dictionary key for the token in the the JSON file. Optional, default is 'token'.</p> <code>'token'</code> <code>auth_token</code> <code>str or None</code> <p>Direct entry of an auth token. If None, uses the file arguments to find the token. Optional, default is None.</p> <code>None</code> <code>max_retries</code> <code>int or None</code> <p>Sets the default number of retries on failed requests. Optional, by default 2.</p> <code>None</code> <code>pool_maxsize</code> <code>int or None</code> <p>Sets the max number of threads in a requests pool, although this value will be exceeded if pool_block is set to False. Optional, uses requests defaults if None.</p> <code>None</code> <code>pool_block</code> <p>If True, prevents the number of threads in a requests pool from exceeding the max size. Optional, uses requests defaults (False) if None.</p> <code>None</code> <code>desired_resolution</code> <code>Iterable[float] or None</code> <p>If given, should be a list or array of the desired resolution you want queries returned in useful for materialization queries.</p> <code>None</code> <code>info_cache</code> <p>Pre-computed info cache, bypassing the lookup of datastack info from the info service. Should only be used in cases where this information is cached and thus repetitive lookups can be avoided.</p> <code>None</code> <code>version</code> <code>Optional[int]</code> <p>The default materialization version of the datastack to use. If None, the latest version is used. Optional, defaults to None.</p> <code>None</code> See Also <p>set_session_defaults</p> <p>get_session_defaults</p> <p>Methods:</p> Name Description <code>change_auth</code> <p>Change the authentication token and reset services.</p> <p>Attributes:</p> Name Type Description <code>annotation</code> <code>AnnotationClient</code> <p>A client for the annotation service. See client.annotation</p> <code>auth</code> <code>AuthClient</code> <p>A client for the auth service. See client.auth for more information.</p> <code>chunkedgraph</code> <code>ChunkedGraphClient</code> <p>A client for the chunkedgraph service. See client.chunkedgraph</p> <code>datastack_name</code> <code>str</code> <p>The name of the datastack for the client.</p> <code>info</code> <code>InfoServiceClient</code> <p>A client for the info service. See client.info for more information.</p> <code>l2cache</code> <code>L2CacheClient</code> <p>A client for the L2 cache service. See client.l2cache</p> <code>materialize</code> <code>MaterializationClient</code> <p>A client for the materialization service. See client.materialize</p> <code>schema</code> <code>SchemaClient</code> <p>A client for the EM Annotation Schemas service. See client.schema</p> <code>server_address</code> <p>The server address for the client.</p> <code>skeleton</code> <code>SkeletonClient</code> <p>A client for the skeleton service. See client.skeleton</p> <code>state</code> <code>JSONService</code> <p>A client for the neuroglancer state service. See client.state</p> <code>timestamp</code> <code>Optional[datetime]</code> <p>The default timestamp to use for queries which rely on a timestamp.</p> <code>version</code> <code>Optional[int]</code> <p>The default materialization version of the datastack to use for queries which</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.annotation","title":"<code>annotation: AnnotationClient</code>  <code>property</code>","text":"<p>A client for the annotation service. See client.annotation for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.auth","title":"<code>auth: AuthClient</code>  <code>property</code>","text":"<p>A client for the auth service. See client.auth for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.chunkedgraph","title":"<code>chunkedgraph: ChunkedGraphClient</code>  <code>property</code>","text":"<p>A client for the chunkedgraph service. See client.chunkedgraph for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.datastack_name","title":"<code>datastack_name: str</code>  <code>property</code>","text":"<p>The name of the datastack for the client.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.info","title":"<code>info: InfoServiceClient</code>  <code>property</code>","text":"<p>A client for the info service. See client.info for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.l2cache","title":"<code>l2cache: L2CacheClient</code>  <code>property</code>","text":"<p>A client for the L2 cache service. See client.l2cache for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.materialize","title":"<code>materialize: MaterializationClient</code>  <code>property</code>","text":"<p>A client for the materialization service. See client.materialize for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.schema","title":"<code>schema: SchemaClient</code>  <code>property</code>","text":"<p>A client for the EM Annotation Schemas service. See client.schema for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.server_address","title":"<code>server_address</code>  <code>property</code>","text":"<p>The server address for the client.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.skeleton","title":"<code>skeleton: SkeletonClient</code>  <code>property</code>","text":"<p>A client for the skeleton service. See client.skeleton for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.state","title":"<code>state: JSONService</code>  <code>property</code>","text":"<p>A client for the neuroglancer state service. See client.state for more information.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.timestamp","title":"<code>timestamp: Optional[datetime]</code>  <code>property</code>","text":"<p>The default timestamp to use for queries which rely on a timestamp.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.version","title":"<code>version: Optional[int]</code>  <code>property</code> <code>writable</code>","text":"<p>The default materialization version of the datastack to use for queries which expect a version. Also sets the timestamp to the corresponding timestamp of the version for queries which rely on a timestamp.</p>"},{"location":"api/client/#caveclient.frameworkclient.CAVEclientFull.change_auth","title":"<code>change_auth(auth_token_file=None, auth_token_key=None, auth_token=None)</code>","text":"<p>Change the authentication token and reset services.</p> <p>Parameters:</p> Name Type Description Default <code>auth_token_file</code> <code>str</code> <p>New auth token json file path, by default None, which defaults to the existing state.</p> <code>None</code> <code>auth_token_key</code> <code>str</code> <p>New dictionary key under which the token is stored in the json file, by default None, which defaults to the existing state.</p> <code>None</code> <code>auth_token</code> <code>str</code> <p>Direct entry of a new token, by default None.</p> <code>None</code>"},{"location":"api/config/","title":"Configuration","text":""},{"location":"api/config/#caveclient.set_session_defaults","title":"<code>caveclient.set_session_defaults(max_retries=3, pool_block=False, pool_maxsize=10, backoff_factor=0.1, backoff_max=120, status_forcelist=(502, 503, 504))</code>","text":"<p>Set global default values to configure how all clients will communicate with servers. Should be done prior to initializing a client.</p> <p>Note that these values are only used when not set at the client level.</p> <p>Parameters:</p> Name Type Description Default <code>max_retries</code> <code>int</code> <p>The maximum number of retries each connection should attempt. Set to 0 to fail on the first retry.</p> <code>3</code> <code>pool_block</code> <code>bool</code> <p>Whether the connection pool should block for connections.</p> <code>False</code> <code>pool_maxsize</code> <code>int</code> <p>The maximum number of connections to save in the pool.</p> <code>10</code> <code>backoff_factor</code> <code>Union[float, int]</code> <p>A backoff factor to apply between attempts after the second try (most errors are resolved immediately by a second try without a delay). The query will sleep for:     <code>{backoff factor} * (2 ^ ({number of total retries} - 1))</code> seconds. For example, if the <code>backoff_factor</code> is 0.1, then will sleep for [0.0s, 0.2s, 0.4s, 0.8s, \u2026] between retries. No backoff will ever be longer than <code>backoff_max</code>.</p> <code>0.1</code> <code>backoff_max</code> <code>Union[float, int]</code> <p>The maximum backoff time.</p> <code>120</code> <code>status_forcelist</code> <code>Optional[Collection]</code> <p>A set of integer HTTP status codes that we should force a retry on.</p> <code>(502, 503, 504)</code> Usage <pre><code>from caveclient import set_session_defaults\n\nset_session_defaults(\n    max_retries=5, # would increase the default number of retries\n    backoff_factor=0.5, # would increase the default backoff factor between retries\n    backoff_max=240, # would increase the default maximum backoff time\n    status_forcelist=(502, 503, 504, 505), # would add 505 to the default list\n)\n\nset_session_defaults() # would revert all defaults to their original values\n</code></pre> Notes <p>Calling this function will set the default values for all clients created after the call.</p> <p>Calling this function with any arguments missing will reset that value to the default value.</p> See Also: <p>urllib3.util.Retry</p> <p>requests.adapters.HTTPAdapter</p>"},{"location":"api/config/#caveclient.get_session_defaults","title":"<code>caveclient.get_session_defaults()</code>","text":"<p>Get the current default values for session configuration.</p> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of current default values for session configuration.</p>"},{"location":"api/datastack_lookup/","title":"Datastack lookup","text":""},{"location":"api/datastack_lookup/#caveclient.datastack_lookup.reset_server_address_cache","title":"<code>caveclient.datastack_lookup.reset_server_address_cache(datastack, filename=None)</code>","text":"<p>Remove one or more datastacks from the datastack-to-server cache.</p> <p>Parameters:</p> Name Type Description Default <code>datastack</code> <code>str or list of str</code> <p>Datastack names to remove from the cache, by default None</p> required <code>filename</code> <code>str</code> <p>Name of the cache file, by default None</p> <code>None</code>"},{"location":"api/info/","title":"client.info","text":"<p>Client for interacting with the info service.</p> <p>Methods:</p> Name Description <code>annotation_endpoint</code> <p>AnnotationEngine endpoint for a dataset.</p> <code>get_aligned_volume_info</code> <p>Gets the info record for a aligned_volume</p> <code>get_datastack_info</code> <p>Gets the info record for a datastack</p> <code>get_datastacks</code> <p>Query which datastacks are available at the info service</p> <code>get_datastacks_by_aligned_volume</code> <p>Lookup what datastacks are associated with this aligned volume</p> <code>image_cloudvolume</code> <p>Generate a cloudvolume instance based on the image source, using authentication if needed and</p> <code>image_source</code> <p>Cloud path to the imagery for the dataset</p> <code>refresh_stored_data</code> <p>Reload the stored info values from the server.</p> <code>segmentation_cloudvolume</code> <p>Generate a cloudvolume instance based on the segmentation source, using authentication if needed and</p> <code>segmentation_source</code> <p>Cloud path to the chunkgraph-backed Graphene segmentation for a dataset</p> <code>synapse_segmentation_source</code> <p>Cloud path to the synapse segmentation for a dataset</p> <code>viewer_resolution</code> <p>Get the viewer resolution metadata for this datastack</p> <code>viewer_site</code> <p>Get the base Neuroglancer URL for the dataset</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.annotation_endpoint","title":"<code>annotation_endpoint(datastack_name=None, use_stored=True)</code>","text":"<p>AnnotationEngine endpoint for a dataset.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <p>Returns:</p> Type Description <code>str</code> <p>Location of the AnnotationEngine</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_aligned_volume_info","title":"<code>get_aligned_volume_info(datastack_name=None, use_stored=True)</code>","text":"<p>Gets the info record for a aligned_volume</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>datastack_name to look up. If None, uses the one specified by the client. By default None</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True and the information has already been queried for that dataset, then uses the cached version. If False, re-queries the infromation. By default True</p> <code>True</code> <p>Returns:</p> Type Description <code>dict or None</code> <p>The complete info record for the aligned_volume</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_datastack_info","title":"<code>get_datastack_info(datastack_name=None, use_stored=True)</code>","text":"<p>Gets the info record for a datastack</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>datastack to look up. If None, uses the one specified by the client. By default None</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True and the information has already been queried for that datastack, then uses the cached version. If False, re-queries the infromation. By default True</p> <code>True</code> <p>Returns:</p> Type Description <code>dict or None</code> <p>The complete info record for the datastack</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_datastacks","title":"<code>get_datastacks()</code>","text":"<p>Query which datastacks are available at the info service</p> <p>Returns:</p> Type Description <code>list</code> <p>List of datastack names</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.get_datastacks_by_aligned_volume","title":"<code>get_datastacks_by_aligned_volume(aligned_volume=None)</code>","text":"<p>Lookup what datastacks are associated with this aligned volume</p> <p>Args:     aligned_volume (str, optional): aligned volume to lookup. Defaults to None.</p> <p>Raises:     ValueError: if no aligned volume is specified</p> <p>Returns:     list: a list of datastack string</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.image_cloudvolume","title":"<code>image_cloudvolume(**kwargs)</code>","text":"<p>Generate a cloudvolume instance based on the image source, using authentication if needed and sensible default values for reading CAVE resources. By default, fill_missing is True and bounded is False. All keyword arguments are passed onto the CloudVolume initialization function, and defaults can be overridden.</p> <p>Requires cloudvolume to be installed, which is not included by default.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.image_source","title":"<code>image_source(datastack_name=None, use_stored=True, format_for='raw')</code>","text":"<p>Cloud path to the imagery for the dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <code>format_for</code> <code>'raw', 'cloudvolume', or 'neuroglancer'</code> <p>Formats the path for different uses. If 'raw' (default), the path in the InfoService is passed along. If 'cloudvolume', a \"precomputed://gs://\" type path is converted to a full https URL. If 'neuroglancer', a full https URL is converted to a \"precomputed://gs://\" type path.</p> <code>'raw'</code> <p>Returns:</p> Type Description <code>str</code> <p>Formatted cloud path to the flat segmentation</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.refresh_stored_data","title":"<code>refresh_stored_data()</code>","text":"<p>Reload the stored info values from the server.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.segmentation_cloudvolume","title":"<code>segmentation_cloudvolume(use_client_secret=True, **kwargs)</code>","text":"<p>Generate a cloudvolume instance based on the segmentation source, using authentication if needed and sensible default values for reading CAVE resources. By default, fill_missing is True and bounded is False. All keyword arguments are passed onto the CloudVolume initialization function, and defaults can be overridden.</p> <p>Requires cloudvolume to be installed, which is not included by default.</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.segmentation_source","title":"<code>segmentation_source(datastack_name=None, format_for='raw', use_stored=True)</code>","text":"<p>Cloud path to the chunkgraph-backed Graphene segmentation for a dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <code>format_for</code> <code>'raw', 'cloudvolume', or 'neuroglancer'</code> <p>Formats the path for different uses. If 'raw' (default), the path in the InfoService is passed along. If 'cloudvolume', a \"graphene://https://\" type path is used If 'neuroglancer', a \"graphene://https://\" type path is used, as needed by Neuroglancer.</p> <code>'raw'</code> <p>Returns:</p> Type Description <code>str</code> <p>Formatted cloud path to the Graphene segmentation</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.synapse_segmentation_source","title":"<code>synapse_segmentation_source(datastack_name=None, use_stored=True, format_for='raw')</code>","text":"<p>Cloud path to the synapse segmentation for a dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the dataset to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <code>format_for</code> <code>'raw', 'cloudvolume', or 'neuroglancer'</code> <p>Formats the path for different uses. If 'raw' (default), the path in the InfoService is passed along. If 'cloudvolume', a \"precomputed://gs://\" type path is converted to a full https URL. If 'neuroglancer', a full https URL is converted to a \"precomputed://gs://\" type path.</p> <code>'raw'</code> <p>Returns:</p> Type Description <code>str</code> <p>Formatted cloud path to the synapse segmentation</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.viewer_resolution","title":"<code>viewer_resolution(datastack_name=None, use_stored=True)</code>","text":"<p>Get the viewer resolution metadata for this datastack</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <p>If None use the default one configured in the client</p> <code>None</code> <code>use_stored</code> <p>Use the cached value, if False go get a new value from server</p> <code>True</code> <p>Returns:</p> Type Description <code>array</code> <p>Voxel resolution as a len(3) np.array</p>"},{"location":"api/info/#caveclient.infoservice.InfoServiceClient.viewer_site","title":"<code>viewer_site(datastack_name=None, use_stored=True)</code>","text":"<p>Get the base Neuroglancer URL for the dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack to look up. If None, uses the value specified by the client. Default is None.</p> <code>None</code> <code>use_stored</code> <code>bool</code> <p>If True, uses the cached value if available. If False, re-queries the InfoService. Default is True.</p> <code>True</code> <p>Returns:</p> Type Description <code>str</code> <p>Base URL for the Neuroglancer viewer</p>"},{"location":"api/l2cache/","title":"client.l2cache","text":"<p>Client for interacting with the level2 cache service.</p> <p>Methods:</p> Name Description <code>cache_metadata</code> <p>Retrieves the meta data for the cache</p> <code>get_l2data</code> <p>Gets the attributed statistics data for L2 ids.</p> <code>get_l2data_table</code> <p>Gets the attributed statistics data for L2 ids, returned as a dataframe.</p> <code>has_cache</code> <p>Checks if the l2 cache is available for the dataset</p> <code>table_mapping</code> <p>Retrieves table mappings for l2 cache.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.cache_metadata","title":"<code>cache_metadata()</code>","text":"<p>Retrieves the meta data for the cache</p> <p>Returns:</p> Type Description <code>dict</code> <p>keys are attribute names, values are datatypes</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.get_l2data","title":"<code>get_l2data(l2_ids, attributes=None)</code>","text":"<p>Gets the attributed statistics data for L2 ids.</p> <p>Parameters:</p> Name Type Description Default <code>l2_ids</code> <code>list or ndarray</code> <p>A list of level 2 ids.</p> required <code>attributes</code> <code>list</code> <p>A list of attributes to retrieve. Defaults to <code>None</code>, which will return all that are available. Available stats are:</p> <ul> <li><code>area_nm2</code></li> <li><code>chunk_intersect_count</code></li> <li><code>max_dt_nm</code></li> <li><code>mean_dt_nm</code></li> <li><code>pca</code></li> <li><code>pca_val</code></li> <li><code>rep_coord_nm</code></li> <li><code>size_nm3</code></li> </ul> <p>See the tutorial for more description.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Keys are l2 ids, values are data.</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.get_l2data_table","title":"<code>get_l2data_table(l2_ids, attributes=None, split_columns=True)</code>","text":"<p>Gets the attributed statistics data for L2 ids, returned as a dataframe.</p> <p>Parameters:</p> Name Type Description Default <code>l2_ids</code> <code>list or ndarray</code> <p>A list of level 2 ids.</p> required <code>attributes</code> <code>list</code> <p>A list of attributes to retrieve. Defaults to <code>None</code>, which will return all that are available. Available stats are:</p> <ul> <li><code>area_nm2</code></li> <li><code>chunk_intersect_count</code></li> <li><code>max_dt_nm</code></li> <li><code>mean_dt_nm</code></li> <li><code>pca</code></li> <li><code>pca_val</code></li> <li><code>rep_coord_nm</code></li> <li><code>size_nm3</code></li> </ul> <p>See the tutorial for more description.</p> <code>None</code> <code>split_columns</code> <code>bool</code> <p>Whether to split columns with multiple values into separate columns.</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe with the requested attributes as columns and indexed by l2_id.</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.has_cache","title":"<code>has_cache(datastack_name=None)</code>","text":"<p>Checks if the l2 cache is available for the dataset</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check, by default None (if None, uses the client's datastack)</p> <code>None</code> <p>Returns:</p> Type Description <code>bool</code> <p>True if the l2 cache is available, False otherwise</p>"},{"location":"api/l2cache/#caveclient.l2cache.L2CacheClient.table_mapping","title":"<code>table_mapping()</code>","text":"<p>Retrieves table mappings for l2 cache.</p> <p>Returns:</p> Type Description <code>dict</code> <p>keys are pcg table names, values are dicts with fields <code>l2cache_id</code> and <code>cv_path</code>.</p>"},{"location":"api/materialize/","title":"client.materialize","text":"<p>Client for interacting with the materialization engine.</p> <p>Methods:</p> Name Description <code>get_table_metadata</code> <p>Get metadata about a table</p> <code>get_tables</code> <p>Gets a list of table names for a datastack</p> <code>get_tables_metadata</code> <p>Get metadata about tables</p> <code>get_timestamp</code> <p>Get datetime.datetime timestamp for a materialization version.</p> <code>get_unique_string_values</code> <p>Get unique string values for a table</p> <code>get_version_metadata</code> <p>Get metadata about a version</p> <code>get_versions</code> <p>Get the versions available</p> <code>get_versions_metadata</code> <p>Get the metadata for all the versions that are presently available and valid</p> <code>get_view_metadata</code> <p>Get metadata for a view</p> <code>get_view_schema</code> <p>Get schema for a view</p> <code>get_view_schemas</code> <p>Get schema for a view</p> <code>get_views</code> <p>Get all available views for a version</p> <code>ingest_annotation_table</code> <p>Trigger supervoxel lookup and root ID lookup of new annotations in a table.</p> <code>join_query</code> <p>Generic query on materialization tables</p> <code>live_live_query</code> <p>Beta method for querying cave annotation tables with root IDs and annotations</p> <code>live_query</code> <p>Generic query on materialization tables</p> <code>lookup_supervoxel_ids</code> <p>Trigger supervoxel lookups of new annotations in a table.</p> <code>map_filters</code> <p>Translate a list of filter dictionaries from a point in the</p> <code>most_recent_version</code> <p>Get the most recent version of materialization for this datastack name</p> <code>query_table</code> <p>Generic query on materialization tables</p> <code>query_view</code> <p>Generic query on a view</p> <code>synapse_query</code> <p>Convenience method for querying synapses.</p> <p>Attributes:</p> Name Type Description <code>cg_client</code> <p>The chunked graph client.</p> <code>datastack_name</code> <p>The name of the datastack.</p> <code>homepage</code> <code>HTML</code> <p>The homepage for the materialization engine.</p> <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p> <code>tables</code> <code>TableManager</code> <p>The table manager for the materialization engine.</p> <code>version</code> <code>int</code> <p>The version of the materialization. Can be used to set up the</p> <code>views</code> <code>ViewManager</code> <p>The view manager for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.cg_client","title":"<code>cg_client</code>  <code>property</code>","text":"<p>The chunked graph client.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.datastack_name","title":"<code>datastack_name</code>  <code>property</code>","text":"<p>The name of the datastack.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.homepage","title":"<code>homepage: HTML</code>  <code>property</code>","text":"<p>The homepage for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.tables","title":"<code>tables: TableManager</code>  <code>property</code>","text":"<p>The table manager for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.version","title":"<code>version: int</code>  <code>property</code> <code>writable</code>","text":"<p>The version of the materialization. Can be used to set up the client to default to a specific version when timestamps or versions are not specified in queries. If not set, defaults to the most recent version.</p> <p>Note that if this materialization client is attached to a CAVEclient, the version must be set at the CAVEclient level.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.views","title":"<code>views: ViewManager</code>  <code>property</code>","text":"<p>The view manager for the materialization engine.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_table_metadata","title":"<code>get_table_metadata(table_name, datastack_name=None, version=None, log_warning=True)</code>","text":"<p>Get metadata about a table</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>name of table to mark for deletion</p> required <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack_name. If None, uses the one specified in the client.</p> <code>None</code> <code>version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to print out warnings to the logger. Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Metadata dictionary for table</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_tables","title":"<code>get_tables(datastack_name=None, version=None)</code>","text":"<p>Gets a list of table names for a datastack</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client</p> <code>None</code> <code>version</code> <code>int or None</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>List of table names</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_tables_metadata","title":"<code>get_tables_metadata(datastack_name=None, version=None, log_warning=True)</code>","text":"<p>Get metadata about tables</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack_name. If None, uses the one specified in the client.</p> <code>None</code> <code>version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to print out warnings to the logger. Defaults to True.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Metadata dictionary for table</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_timestamp","title":"<code>get_timestamp(version=None, datastack_name=None)</code>","text":"<p>Get datetime.datetime timestamp for a materialization version.</p> <p>Parameters:</p> Name Type Description Default <code>version</code> <code>int or None</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>datastack_name</code> <code>str or None</code> <p>Datastack name, by default None. If None, defaults to the value set in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>datetime</code> <p>Datetime when the materialization version was frozen.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_unique_string_values","title":"<code>get_unique_string_values(table, datastack_name=None)</code>","text":"<p>Get unique string values for a table</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Table to query</p> required <code>datastack_name</code> <code>Optional[str]</code> <p>Datastack to query. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict[str]</code> <p>A dictionary of column names and their unique values</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_version_metadata","title":"<code>get_version_metadata(version=None, datastack_name=None)</code>","text":"<p>Get metadata about a version</p> <p>Parameters:</p> Name Type Description Default <code>version</code> <code>int or None</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>datastack_name</code> <code>str or None</code> <p>Datastack name, by default None. If None, defaults to the value set in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of metadata about the version</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_versions","title":"<code>get_versions(datastack_name=None, expired=False)</code>","text":"<p>Get the versions available</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack, by default None. If None, uses the one specified in the client.</p> <code>None</code> <code>expired</code> <code>bool</code> <p>Whether to include expired versions, by default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of versions available</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_versions_metadata","title":"<code>get_versions_metadata(datastack_name=None, expired=False)</code>","text":"<p>Get the metadata for all the versions that are presently available and valid</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Datastack name, by default None. If None, defaults to the value set in the client.</p> <code>None</code> <code>expired</code> <code>bool</code> <p>Whether to include expired versions, by default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>list[dict]</code> <p>List of metadata dictionaries</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_view_metadata","title":"<code>get_view_metadata(view_name, materialization_version=None, datastack_name=None, log_warning=True)</code>","text":"<p>Get metadata for a view</p> <p>Parameters:</p> Name Type Description Default <code>view_name</code> <code>str</code> <p>Name of view to query.</p> required <code>materialization_version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Metadata of view</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_view_schema","title":"<code>get_view_schema(view_name, materialization_version=None, datastack_name=None, log_warning=True)</code>","text":"<p>Get schema for a view</p> <p>Parameters:</p> Name Type Description Default <code>view_name</code> <code>str</code> <p>Name of view to query.</p> required <code>materialization_version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Schema of view.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_view_schemas","title":"<code>get_view_schemas(materialization_version=None, datastack_name=None, log_warning=True)</code>","text":"<p>Get schema for a view</p> <p>Parameters:</p> Name Type Description Default <code>materialization_version</code> <code>Optional[int]</code> <p>Version to query. If None, will use version set by client.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>dict</code> <p>Schema of view.</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.get_views","title":"<code>get_views(version=None, datastack_name=None)</code>","text":"<p>Get all available views for a version</p> <p>Parameters:</p> Name Type Description Default <code>version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query. If None, uses the one specified in the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>list</code> <p>List of views</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.ingest_annotation_table","title":"<code>ingest_annotation_table(table_name, datastack_name=None)</code>","text":"<p>Trigger supervoxel lookup and root ID lookup of new annotations in a table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Table to trigger</p> required <code>datastack_name</code> <code>str</code> <p>Datastack to trigger it. Defaults to what is set in client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Status code of response from server</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.join_query","title":"<code>join_query(tables, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, suffixes=None, datastack_name=None, return_df=True, split_positions=False, materialization_version=None, metadata=True, desired_resolution=None, random_sample=None, log_warning=True)</code>","text":"<p>Generic query on materialization tables</p> <p>Parameters:</p> Name Type Description Default <code>tables</code> <code>list of lists with length 2 or 'str'</code> <p>list of two lists: first entries are table names, second entries are the columns used for the join.</p> required <code>filter_in_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are allowed entries, by default None</p> <code>None</code> <code>filter_out_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are not allowed entries, by default None</p> <code>None</code> <code>filter_equal_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are specified entry, by default None</p> <code>None</code> <code>filter_greater_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are exclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are exclusive lower-bound, by default None</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are inclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names inner layer: keys are column names, values are inclusive lower-bound, by default None</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names, inner layer: keys are column names. Values are bounding boxes as [[min_x, min_y,min_z],[max_x, max_y, max_z]], expressed in units of the voxel_resolution of this dataset. Defaults to None.</p> <code>None</code> <code>filter_regex_dict</code> <code>dict of dicts</code> <p>outer layer: keys are table names. inner layer: keys are column names, values are regex strings. Defaults to None</p> <code>None</code> <code>select_columns</code> <code>dict of lists of str</code> <p>keys are table names,values are the list of columns from that table. Defaults to None, which will select all tables.  Will be passed to server as select_column_maps. Passing a list will be passed as select_columns which is deprecated.</p> <code>None</code> <code>offset</code> <code>int</code> <p>result offset to use. Defaults to None. Will only return top K results.</p> <code>None</code> <code>limit</code> <code>int</code> <p>maximum results to return (server will set upper limit, see get_server_config)</p> <code>None</code> <code>suffixes</code> <code>dict</code> <p>suffixes to use for duplicate columns, keys are table names, values are the suffix</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>datastack to query. If None defaults to one specified in client.</p> <code>None</code> <code>return_df</code> <code>bool</code> <p>whether to return as a dataframe default True, if False, data is returned as json (slower)</p> <code>True</code> <code>split_positions</code> <code>bool</code> <p>whether to break position columns into x,y,z columns default False, if False data is returned as one column with [x,y,z] array (slower)</p> <code>False</code> <code>materialization_version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>toggle to return metadata If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>desired_resolution</code> <code>Iterable</code> <p>What resolution to convert position columns to. Defaults to None will use defaults.</p> <code>None</code> <code>random_sample</code> <code>int</code> <p>if given, will do a tablesample of the table to return that many annotations</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings, by default True</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>a pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.live_live_query","title":"<code>live_live_query(table, timestamp, joins=None, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, split_positions=False, metadata=True, suffixes=None, desired_resolution=None, allow_missing_lookups=False, allow_invalid_root_ids=False, random_sample=None, log_warning=True)</code>","text":"<p>Beta method for querying cave annotation tables with root IDs and annotations at a particular timestamp.  Note: this method requires more explicit mapping of filters and selection to table as its designed to test a more general endpoint that should eventually support complex joins.</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Principle table to query</p> required <code>timestamp</code> <code>datetime</code> <p>Timestamp to query</p> required <code>joins</code> <p>List of joins, where each join is a list of [table1,column1, table2, column2]</p> <code>None</code> <code>filter_in_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and list values to accept.</p> <code>None</code> <code>filter_out_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and list values to reject.</p> <code>None</code> <code>filter_equal_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values to equate.</p> <code>None</code> <code>filter_greater_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as exclusive upper-bound.</p> <code>None</code> <code>filter_less_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as exclusive lower-bound.</p> <code>None</code> <code>filter_greater_equal_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as inclusive upper-bound.</p> <code>None</code> <code>filter_less_equal_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values as inclusive lower-bound.</p> <code>None</code> <code>filter_spatial_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values of 2x3 list of bounds.</p> <code>None</code> <code>filter_regex_dict</code> <p>A dictionary with tables as keys, values are dicts with column keys and values of regex strings.</p> <code>None</code> <code>select_columns</code> <p>A dictionary with tables as keys, values are lists of columns to select.</p> <code>None</code> <code>offset</code> <code>int</code> <p>Value to offset query by.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Limit of query.</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query. Defaults to set by client.</p> <code>None</code> <code>split_positions</code> <code>bool</code> <p>Whether to split positions into separate columns, True is faster.</p> <code>False</code> <code>metadata</code> <code>bool</code> <p>Whether to attach metadata to dataframe.</p> <code>True</code> <code>suffixes</code> <code>dict</code> <p>What suffixes to use on joins, keys are table_names, values are suffixes.</p> <code>None</code> <code>desired_resolution</code> <code>Iterable</code> <p>What resolution to convert position columns to.</p> <code>None</code> <code>allow_missing_lookups</code> <code>bool</code> <p>If there are annotations without supervoxels and root IDs yet, allow results.</p> <code>False</code> <code>allow_invalid_root_ids</code> <code>bool</code> <p>If True, ignore root ids not valid at the given timestamp, otherwise raise an error.</p> <code>False</code> <code>random_sample</code> <code>int</code> <p>If given, will do a table sample of the table to return that many annotations.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <p>Results of query</p> <p>Examples:</p> <pre><code>&gt;&gt;&gt; from caveclient import CAVEclient\n&gt;&gt;&gt; client = CAVEclient('minnie65_public_v117')\n&gt;&gt;&gt; live_live_query(\"table_name\", datetime.datetime.now(datetime.timezone.utc),\n&gt;&gt;&gt;    joins=[[table_name, table_column, joined_table, joined_column],\n&gt;&gt;&gt;             [joined_table, joincol2, third_table, joincol_third]]\n&gt;&gt;&gt;    suffixes={\n&gt;&gt;&gt;        \"table_name\":\"suffix1\",\n&gt;&gt;&gt;        \"joined_table\":\"suffix2\",\n&gt;&gt;&gt;        \"third_table\":\"suffix3\"\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    select_columns= {\n&gt;&gt;&gt;        \"table_name\":[ \"column\",\"names\"],\n&gt;&gt;&gt;        \"joined_table\":[\"joined_colum\"]\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    filter_in_dict= {\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":[included,values]\n&gt;&gt;&gt;        }\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    filter_out_dict= {\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":[excluded,values]\n&gt;&gt;&gt;        }\n&gt;&gt;&gt;    },\n&gt;&gt;&gt;    filter_equal_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_greater_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_less_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_greater_equal_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_less_equal_dict\"={\n&gt;&gt;&gt;        \"table_name\":{\n&gt;&gt;&gt;            \"column_name\":value\n&gt;&gt;&gt;        },\n&gt;&gt;&gt;    filter_spatial_dict\"= {\n&gt;&gt;&gt;        \"table_name\": {\n&gt;&gt;&gt;        \"column_name\": [[min_x, min_y, min_z], [max_x, max_y, max_z]]\n&gt;&gt;&gt;    }\n&gt;&gt;&gt;    filter_regex_dict\"= {\n&gt;&gt;&gt;        \"table_name\": {\n&gt;&gt;&gt;        \"column_name\": \"regex_string\"\n&gt;&gt;&gt;     }\n</code></pre>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.live_query","title":"<code>live_query(table, timestamp, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, split_positions=False, post_filter=True, metadata=True, merge_reference=True, desired_resolution=None, random_sample=None, log_warning=True)</code>","text":"<p>Generic query on materialization tables</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Table to query</p> required <code>timestamp</code> <code>datetime</code> <p>Time to materialize (in utc). Pass datetime.datetime.now(datetime.timezone.utc) for present time.</p> required <code>filter_in_dict</code> <code>dict</code> <p>Keys are column names, values are allowed entries.</p> <code>None</code> <code>filter_out_dict</code> <code>dict</code> <p>Keys are column names, values are not allowed entries.</p> <code>None</code> <code>filter_equal_dict</code> <code>dict</code> <p>Keys are column names, values are specified entry.</p> <code>None</code> <code>filter_greater_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive upper-bounds.</p> <code>None</code> <code>filter_less_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive lower-bounds.</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive upper-bounds.</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive lower-bounds.</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict</code> <p>Keys are column names, values are bounding boxes expressed in units of the voxel_resolution of this dataset. Bounding box is [[min_x, min_y,min_z],[max_x, max_y, max_z]].</p> <code>None</code> <code>filter_regex_dict</code> <code>dict</code> <p>Keys are column names, values are regex strings.</p> <code>None</code> <code>select_columns</code> <code>list of str</code> <p>Columns to select.</p> <code>None</code> <code>offset</code> <code>int</code> <p>Offset in query result.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Maximum results to return (server will set upper limit, see get_server_config).</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query. If None, defaults to one specified in client.</p> <code>None</code> <code>split_positions</code> <code>bool</code> <p>Whether to break position columns into x,y,z columns. If False data is returned as one column with [x,y,z] array (slower).</p> <code>False</code> <code>post_filter</code> <code>bool</code> <p>Whether to filter down the result based upon the filters specified. If False, it will return the query with present root_ids in the root_id columns, but the rows will reflect the filters translated into their past IDs. So if, for example, a cell had a false merger split off since the last materialization, those annotations on that incorrect portion of the cell will be included if this is False, but will be filtered down if this is True.</p> <code>True</code> <code>metadata</code> <code>bool</code> <p>Toggle to return metadata. If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>merge_reference</code> <code>bool</code> <p>Toggle to automatically join reference table. If True, metadata will be queries and if its a reference table it will perform a join on the reference table to return the rows of that table.</p> <code>True</code> <code>desired_resolution</code> <code>Iterable</code> <p>Desired resolution you want all spatial points returned in. If None, defaults to one specified in client, if that is None then points are returned as stored in the table and should be in the resolution specified in the table metadata.</p> <code>None</code> <code>random_sample</code> <code>int</code> <p>If given, will do a tablesample of the table to return that many annotations.</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings.</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.lookup_supervoxel_ids","title":"<code>lookup_supervoxel_ids(table_name, annotation_ids=None, datastack_name=None)</code>","text":"<p>Trigger supervoxel lookups of new annotations in a table.</p> <p>Parameters:</p> Name Type Description Default <code>table_name</code> <code>str</code> <p>Table to trigger</p> required <code>annotation_ids</code> <code>list</code> <p>List of annotation ids to lookup. Default is None, which will trigger lookup of entire table.</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to trigger it. Defaults to what is set in client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>Status code of response from server</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.map_filters","title":"<code>map_filters(filters, timestamp, timestamp_past)</code>","text":"<p>Translate a list of filter dictionaries from a point in the future to a point in the past</p> <p>Parameters:</p> Name Type Description Default <code>filters</code> <code>list[dict]</code> <p>filter dictionaries with root_ids</p> required <code>timestamp</code> <code>datetime</code> <p>timestamp to query</p> required <code>timestamp_past</code> <code>datetime</code> <p>timestamp to query from</p> required <p>Returns:</p> Type Description <code>list[dict]</code> <p>filter dictionaries with past root_ids</p> <code>dict</code> <p>mapping of future root_ids to past root_ids</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.most_recent_version","title":"<code>most_recent_version(datastack_name=None)</code>","text":"<p>Get the most recent version of materialization for this datastack name</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str or None</code> <p>Name of the datastack, by default None. If None, uses the one specified in the client. Will be set correctly if you are using the framework_client</p> <code>None</code> <p>Returns:</p> Type Description <code>int</code> <p>Most recent version of materialization for this datastack name</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.query_table","title":"<code>query_table(table, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, return_df=True, split_positions=False, materialization_version=None, timestamp=None, metadata=True, merge_reference=True, desired_resolution=None, get_counts=False, random_sample=None, log_warning=True)</code>","text":"<p>Generic query on materialization tables</p> <p>Parameters:</p> Name Type Description Default <code>table</code> <code>str</code> <p>Table to query</p> required <code>filter_in_dict</code> <code>dict</code> <p>Keys are column names, values are allowed entries, by default None</p> <code>None</code> <code>filter_out_dict</code> <code>dict</code> <p>Keys are column names, values are not allowed entries, by default None</p> <code>None</code> <code>filter_equal_dict</code> <code>dict</code> <p>Keys are column names, values are specified entry, by default None</p> <code>None</code> <code>filter_greater_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive upper-bound value, by default None</p> <code>None</code> <code>filter_less_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive lower-bound value, by default None</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive upper-bound value, by default None</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive lower-bound value, by default None</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict</code> <p>Keys are column names, values are bounding boxes expressed in units of the voxel_resolution of this dataset. Bounding box is [[min_x, min_y,min_z],[max_x, max_y, max_z]], by default None</p> <code>None</code> <code>filter_regex_dict</code> <code>dict</code> <p>Keys are column names, values are regex strings, by default None</p> <code>None</code> <code>select_columns</code> <code>list of str</code> <p>Columns to select, by default None</p> <code>None</code> <code>offset</code> <code>int</code> <p>Result offset to use, by default None. Will only return top K results.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Maximum results to return (server will set upper limit, see get_server_config), by default None</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query, by default None. If None, defaults to one specified in client.</p> <code>None</code> <code>return_df</code> <code>bool</code> <p>Whether to return as a dataframe, by default True. If False, data is returned as json (slower).</p> <code>True</code> <code>split_positions</code> <code>bool</code> <p>Whether to break position columns into x,y,z columns, by default False. If False data is returned as one column with [x,y,z] array (slower)</p> <code>False</code> <code>materialization_version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>timestamp</code> <code>datetime</code> <p>Timestamp to query, by default None. If passsed will do a live query. Error if also passing a materialization version</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>Toggle to return metadata (default True), by default True. If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>merge_reference</code> <code>bool</code> <p>Toggle to automatically join reference table, by default True. If True, metadata will be queries and if its a reference table it will perform a join on the reference table to return the rows of that</p> <code>True</code> <code>desired_resolution</code> <code>Iterable[float]</code> <p>Desired resolution you want all spatial points returned in, by default None. If None, defaults to one specified in client, if that is None then points are returned as stored in the table and should be in the resolution specified in the table metadata</p> <code>None</code> <code>get_counts</code> <code>bool</code> <p>Whether to get counts of the query, by default False</p> <code>False</code> <code>random_sample</code> <code>int</code> <p>If given, will do a tablesample of the of the table to return that many annotations</p> <code>None</code> <code>log_warning</code> <code>bool</code> <p>Whether to log warnings, by default True</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.query_view","title":"<code>query_view(view_name, filter_in_dict=None, filter_out_dict=None, filter_equal_dict=None, filter_greater_dict=None, filter_less_dict=None, filter_greater_equal_dict=None, filter_less_equal_dict=None, filter_spatial_dict=None, filter_regex_dict=None, select_columns=None, offset=None, limit=None, datastack_name=None, return_df=True, split_positions=False, materialization_version=None, metadata=True, merge_reference=True, desired_resolution=None, get_counts=False, random_sample=None)</code>","text":"<p>Generic query on a view</p> <p>Parameters:</p> Name Type Description Default <code>view_name</code> <code>str</code> <p>View to query</p> required <code>filter_in_dict</code> <code>dict</code> <p>Keys are column names, values are allowed entries, by default None</p> <code>None</code> <code>filter_out_dict</code> <code>dict</code> <p>Keys are column names, values are not allowed entries, by default None</p> <code>None</code> <code>filter_equal_dict</code> <code>dict</code> <p>Keys are column names, values are specified entry, by default None</p> <code>None</code> <code>filter_greater_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_dict</code> <code>dict</code> <p>Keys are column names, values are exclusive lower-bound, by default None</p> <code>None</code> <code>filter_greater_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive upper-bound, by default None</p> <code>None</code> <code>filter_less_equal_dict</code> <code>dict</code> <p>Keys are column names, values are inclusive lower-bound, by default None</p> <code>None</code> <code>filter_spatial_dict</code> <code>dict</code> <p>Keys are column names, values are bounding boxes expressed in units of the voxel_resolution of this dataset. Bounding box is [[min_x, min_y,min_z],[max_x, max_y, max_z]], by default None</p> <code>None</code> <code>filter_regex_dict</code> <code>dict</code> <p>Keys are column names, values are regex strings, by default None</p> <code>None</code> <code>select_columns</code> <code>list of str</code> <p>Columns to select, by default None</p> <code>None</code> <code>offset</code> <code>int</code> <p>Result offset to use, by default None. Will only return top K results.</p> <code>None</code> <code>limit</code> <code>int</code> <p>Maximum results to return (server will set upper limit, see get_server_config), by default None</p> <code>None</code> <code>datastack_name</code> <code>str</code> <p>Datastack to query, by default None. If None, defaults to one specified in client.</p> <code>None</code> <code>return_df</code> <code>bool</code> <p>Whether to return as a dataframe, by default True. If False, data is returned as json (slower).</p> <code>True</code> <code>split_positions</code> <code>bool</code> <p>Whether to break position columns into x,y,z columns, by default False. If False data is returned as one column with [x,y,z] array (slower)</p> <code>False</code> <code>materialization_version</code> <code>int</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>Toggle to return metadata (default True), by default True. If True (and return_df is also True), return table and query metadata in the df.attr dictionary.</p> <code>True</code> <code>merge_reference</code> <code>bool</code> <p>Toggle to automatically join reference table, by default True. If True, metadata will be queries and if its a reference table it will perform a join on the reference table to return the rows of that</p> <code>True</code> <code>desired_resolution</code> <code>Iterable[float]</code> <p>Desired resolution you want all spatial points returned in, by default None. If None, defaults to one specified in client, if that is None then points are returned as stored in the table and should be in the resolution specified in the table metadata</p> <code>None</code> <code>get_counts</code> <code>bool</code> <p>Whether to get counts of the query, by default False</p> <code>False</code> <code>random_sample</code> <code>int</code> <p>If given, will do a tablesample of the of the table to return that many annotations</p> <code>None</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>A pandas dataframe of results of query</p>"},{"location":"api/materialize/#caveclient.materializationengine.MaterializationClient.synapse_query","title":"<code>synapse_query(pre_ids=None, post_ids=None, bounding_box=None, bounding_box_column='post_pt_position', timestamp=None, remove_autapses=True, include_zeros=True, limit=None, offset=None, split_positions=False, desired_resolution=None, materialization_version=None, synapse_table=None, datastack_name=None, metadata=True)</code>","text":"<p>Convenience method for querying synapses.</p> <p>Will use the synapse table specified in the info service by default. It will also remove autapses by default. NOTE: This is not designed to allow querying of the entire synapse table. A query with no filters will return only a limited number of rows (configured by the server) and will do so in a non-deterministic fashion. Please contact your dataset administrator if you want access to the entire table.</p> <p>Parameters:</p> Name Type Description Default <code>pre_ids</code> <code>Union[int, Iterable, ndarray]</code> <p>Pre-synaptic cell(s) to query.</p> <code>None</code> <code>post_ids</code> <code>Union[int, Iterable, ndarray]</code> <p>Post-synaptic cell(s) to query.</p> <code>None</code> <code>bounding_box</code> <code>Optional[Union[list, ndarray]]</code> <p>[[min_x, min_y, min_z],[max_x, max_y, max_z]] bounding box to filter synapse locations. Expressed in units of the voxel_resolution of this dataset.</p> <code>None</code> <code>bounding_box_column</code> <code>str</code> <p>Which synapse location column to filter by.</p> <code>'post_pt_position'</code> <code>timestamp</code> <code>datetime</code> <p>Timestamp to query. If passed recalculate query at timestamp, do not pass with materialization_version.</p> <code>None</code> <code>remove_autapses</code> <code>bool</code> <p>Whether to remove autapses from query results.</p> <code>True</code> <code>include_zeros</code> <code>bool</code> <p>Whether to include synapses to/from id=0 (out of segmentation).</p> <code>True</code> <code>limit</code> <code>int</code> <p>Number of synapses to limit. Server-side limit still applies.</p> <code>None</code> <code>offset</code> <code>int</code> <p>Number of synapses to offset query.</p> <code>None</code> <code>split_positions</code> <code>bool</code> <p>Whether to split positions into separate columns, True is faster.</p> <code>False</code> <code>desired_resolution</code> <code>Iterable[float]</code> <p>List or array of the desired resolution you want queries returned in useful for materialization queries.</p> <code>None</code> <code>materialization_version</code> <code>Optional[int]</code> <p>The version of the datastack to query. If None, will query the client <code>version</code>, which defaults to the most recent version.</p> <code>None</code> <code>metadata</code> <code>bool</code> <p>Whether to attach metadata to dataframe in the df.attr dictionary.</p> <code>True</code> <p>Returns:</p> Type Description <code>DataFrame</code> <p>Results of query.</p>"},{"location":"api/schema/","title":"client.schema","text":"<p>Client for interacting with the schema service.</p> <p>Methods:</p> Name Description <code>get_schemas</code> <p>Get the available schema types</p> <code>schema_definition</code> <p>Get the definition of a specified schema_type</p> <code>schema_definition_all</code> <p>Get the definition of all schema_types</p> <code>schema_definition_multi</code> <p>Get the definition of multiple schema_types</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.get_schemas","title":"<code>get_schemas()</code>","text":"<p>Get the available schema types</p> <p>Returns:</p> Type Description <code>list</code> <p>List of schema types available on the Schema service.</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.schema_definition","title":"<code>schema_definition(schema_type)</code>","text":"<p>Get the definition of a specified schema_type</p> <p>Parameters:</p> Name Type Description Default <code>schema_type</code> <code>str</code> <p>Name of a schema_type</p> required <p>Returns:</p> Type Description <code>json</code> <p>Schema definition</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.schema_definition_all","title":"<code>schema_definition_all()</code>","text":"<p>Get the definition of all schema_types</p> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of schema definitions. Keys are schema names, values are definitions.</p>"},{"location":"api/schema/#caveclient.emannotationschemas.SchemaClient.schema_definition_multi","title":"<code>schema_definition_multi(schema_types)</code>","text":"<p>Get the definition of multiple schema_types</p> <p>Parameters:</p> Name Type Description Default <code>schema_types</code> <code>list</code> <p>List of schema names</p> required <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary of schema definitions. Keys are schema names, values are definitions.</p>"},{"location":"api/skeleton/","title":"client.skeleton","text":"<p>Client for interacting with the skeleton service.</p> <p>Methods:</p> Name Description <code>get_skeleton</code> <p>Gets basic skeleton information for a datastack</p> <code>get_cache_contents</code> <p>Mirror CloudFiles.list() for skeletons as a pass-through interface to the underlying service and bucket.</p> <code>skeletons_exist</code> <p>Confirm or deny that a set of root ids have H5 skeletons in the cache.</p> <code>get_bulk_skeletons</code> <p>Generates skeletons for a list of root ids without retrieving them.</p> <code>generate_bulk_skeletons_async</code> <p>Generates skeletons for a list of root ids without retrieving them.</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.get_skeleton","title":"<code>get_skeleton(root_id, datastack_name=None, skeleton_version=3, output_format='dict', log_warning=True, verbose_level=0)</code>","text":"<p>Gets basic skeleton information for a datastack</p> <p>Parameters:</p> Name Type Description Default <code>root_id</code> <code>int</code> <p>The root id of the skeleton to retrieve</p> required <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check</p> <code>None</code> <code>skeleton_version</code> <code>int</code> <p>The skeleton version to generate and retrieve. Options are documented in SkeletonService. Use 0 for Neuroglancer-compatibility. Use -1 for latest.</p> <code>3</code> <code>output_format</code> <code>string</code> <p>The format to retrieve. Options are: - 'dict': A dictionary - 'swc': A pandas DataFrame</p> <code>'dict'</code> <p>Returns:</p> Type Description <p>Skeleton of the requested type. See <code>output_format</code> for details.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.get_cache_contents","title":"<code>get_cache_contents(datastack_name=None, skeleton_version=3, root_id_prefixes=0, limit=0, log_warning=True)</code>","text":"<p>Mirror CloudFiles.list() for skeletons as a pass-through interface to the underlying service and bucket.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.skeletons_exist","title":"<code>skeletons_exist(datastack_name=None, skeleton_version=3, root_ids=0, log_warning=True)</code>","text":"<p>Confirm or deny that a set of root ids have H5 skeletons in the cache.</p>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.get_bulk_skeletons","title":"<code>get_bulk_skeletons(root_ids, datastack_name=None, skeleton_version=3, output_format='dict', generate_missing_skeletons=False, log_warning=True, verbose_level=0)</code>","text":"<p>Generates skeletons for a list of root ids without retrieving them.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>List</code> <p>A list of root ids of the skeletons to generate</p> required <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check</p> <code>None</code> <code>skeleton_version</code> <code>int</code> <p>The skeleton version to generate. Use 0 for Neuroglancer-compatibility. Use -1 for latest.</p> <code>3</code>"},{"location":"api/skeleton/#caveclient.skeletonservice.SkeletonClient.generate_bulk_skeletons_async","title":"<code>generate_bulk_skeletons_async(root_ids, datastack_name=None, skeleton_version=None, log_warning=True, verbose_level=0)</code>","text":"<p>Generates skeletons for a list of root ids without retrieving them.</p> <p>Parameters:</p> Name Type Description Default <code>root_ids</code> <code>List</code> <p>A list of root ids of the skeletons to generate</p> required <code>datastack_name</code> <code>str</code> <p>The name of the datastack to check</p> <code>None</code> <code>skeleton_version</code> <code>int</code> <p>The skeleton version to generate. Use 0 for Neuroglancer-compatibility. Use -1 for latest.</p> <code>None</code> <p>Returns:</p> Type Description <code>float</code> <p>The estimated time in seconds to generate all skeletons (a comparable message will be output to the console prior to return).</p>"},{"location":"api/state/","title":"client.state","text":"<p>Client to interface with the JSON state service.</p> <p>Methods:</p> Name Description <code>build_neuroglancer_url</code> <p>Build a URL for a Neuroglancer deployment that will automatically retrieve specified state.</p> <code>get_neuroglancer_info</code> <p>Get the info field from a Neuroglancer deployment</p> <code>get_property_json</code> <p>Download a Neuroglancer JSON state</p> <code>get_state_json</code> <p>Download a Neuroglancer JSON state</p> <code>save_state_json_local</code> <p>Save a Neuroglancer JSON state to a JSON file locally.</p> <code>upload_property_json</code> <p>Upload a Neuroglancer JSON state</p> <code>upload_state_json</code> <p>Upload a Neuroglancer JSON state</p> <p>Attributes:</p> Name Type Description <code>server_version</code> <code>Optional[Version]</code> <p>The version of the service running on the remote server. Note that this</p> <code>state_service_endpoint</code> <code>str</code> <p>Endpoint URL for posting JSON state</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.server_version","title":"<code>server_version: Optional[Version]</code>  <code>property</code>","text":"<p>The version of the service running on the remote server. Note that this refers to the software running on the server and has nothing to do with the version of the datastack itself.</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.state_service_endpoint","title":"<code>state_service_endpoint: str</code>  <code>property</code>","text":"<p>Endpoint URL for posting JSON state</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.build_neuroglancer_url","title":"<code>build_neuroglancer_url(state_id, ngl_url=None, target_site=None, static_url=False, format_properties=False)</code>","text":"<p>Build a URL for a Neuroglancer deployment that will automatically retrieve specified state. If the datastack is specified, this is prepopulated from the info file field \"viewer_site\". If no ngl_url is specified in either the function or the client, a fallback neuroglancer deployment is used.</p> <p>Parameters:</p> Name Type Description Default <code>state_id</code> <code>int</code> <p>State id to retrieve</p> required <code>ngl_url</code> <code>str</code> <p>Base url of a neuroglancer deployment. If None, defaults to the value for the datastack or the client. As a fallback, a default deployment is used.</p> <code>None</code> <code>target_site</code> <code>seunglab or cave - explorer or mainline or None</code> <p>Set this to 'seunglab' for a seunglab deployment, or either 'cave-explorer'/'mainline' for a google main branch deployment. If None, checks the info field of the neuroglancer endpoint to determine which to use. Default is None.</p> <code>None</code> <code>static_url</code> <code>bool</code> <p>If True, treats \"state_id\" as a static URL directly to the JSON and does not use the state service.</p> <code>False</code> <code>format_properties</code> <code>bool</code> <p>If True, formats the url as a segment_properties info file</p> <code>False</code> <p>Returns:</p> Type Description <code>str</code> <p>The full URL requested</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.get_neuroglancer_info","title":"<code>get_neuroglancer_info(ngl_url=None)</code>","text":"<p>Get the info field from a Neuroglancer deployment</p> <p>Parameters:</p> Name Type Description Default <code>ngl_url</code> <code>str(optional)</code> <p>URL to a Neuroglancer deployment. If None, defaults to the value for the datastack or the client.</p> <code>None</code> <p>Returns:</p> Type Description <code>dict</code> <p>JSON-formatted info field from the Neuroglancer deployment</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.get_property_json","title":"<code>get_property_json(state_id)</code>","text":"<p>Download a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>state_id</code> <code>int</code> <p>ID of a JSON state uploaded to the state service.</p> required <p>Returns:</p> Type Description <code>dict</code> <p>JSON specifying a Neuroglancer state.</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.get_state_json","title":"<code>get_state_json(state_id)</code>","text":"<p>Download a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>state_id</code> <code>int or string</code> <p>ID of a JSON state uploaded to the state service. If a string, treats this as the url to the json state.</p> required <p>Returns:</p> Type Description <code>dict</code> <p>JSON specifying a Neuroglancer state.</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.save_state_json_local","title":"<code>save_state_json_local(json_state, filename, overwrite=False)</code>","text":"<p>Save a Neuroglancer JSON state to a JSON file locally.</p> <p>Parameters:</p> Name Type Description Default <code>json_state</code> <code>dict</code> <p>Dict representation of a neuroglancer state</p> required <code>filename</code> <code>str</code> <p>Filename to save the state to</p> required <code>overwrite</code> <code>bool</code> <p>Whether to overwrite the file if it exists. Default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>None</code>"},{"location":"api/state/#caveclient.jsonservice.JSONService.upload_property_json","title":"<code>upload_property_json(property_json, state_id=None, max_size=2500000)</code>","text":"<p>Upload a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>propery_json</code> <code>dict</code> <p>Dict representation of a neuroglancer segment properties json</p> required <code>state_id</code> <code>int</code> <p>ID of a JSON state uploaded to the state service. Using a state_id is an admin feature.</p> <code>None</code> <code>max_size</code> <code>Optional[int]</code> <p>Maximum size in bytes for the data to upload. Default is 2.5MB. Set to None for no limit.</p> <code>2500000</code> <p>Returns:</p> Type Description <code>int</code> <p>state_id of the uploaded JSON state</p>"},{"location":"api/state/#caveclient.jsonservice.JSONService.upload_state_json","title":"<code>upload_state_json(json_state, state_id=None, timestamp=None)</code>","text":"<p>Upload a Neuroglancer JSON state</p> <p>Parameters:</p> Name Type Description Default <code>json_state</code> <code>dict</code> <p>Dict representation of a neuroglancer state</p> required <code>state_id</code> <code>int</code> <p>ID of a JSON state uploaded to the state service. Using a state_id is an admin feature.</p> <code>None</code> <code>timestamp</code> <code>Optional[time]</code> <p>Timestamp for json state date. Requires state_id.</p> <code>None</code> <p>Returns:</p> Type Description <code>int</code> <p>state_id of the uploaded JSON state</p>"},{"location":"api/testing/","title":"caveclient.tools.testing","text":"<p>Functions:</p> Name Description <code>CAVEclientMock</code> <p>Created a mocked CAVEclient function for testing using the responses library to mock</p> <code>get_materialiation_info</code> <p>Get the materialization versions and version metadata for the materialization service.</p> <code>get_server_versions</code> <p>Get the server versions for the services used in testing.</p> <code>get_server_information</code> <p>Generate the datastack name and server locations used in testing.</p> <code>get_api_versions</code> <p>Get the API versions for the services used in testing.</p> <code>default_info</code> <p>Generate a info service info file for testing</p>"},{"location":"api/testing/#caveclient.tools.testing.CAVEclientMock","title":"<code>CAVEclientMock(datastack_name=None, global_server=None, local_server=None, info_file=None, chunkedgraph=False, chunkedgraph_server_version=DEFAULT_CHUNKEDGRAPH_SERVER_VERSION, chunkedgraph_api_versions=None, materialization=False, materialization_server_version=DEFAULT_MATERIALIZATION_SERVER_VERSON, materialization_api_versions=None, available_materialization_versions=None, set_version=None, set_version_metadata=None, json_service=False, json_service_server_version=DEFAULT_JSON_SERVICE_SERVER_VERSION, skeleton_service=False, skeleton_service_server_version=DEFAULT_SKELETON_SERVICE_SERVER_VERSION, schema_api_versions=None, l2cache=False, l2cache_disabled=False, global_only=False)</code>","text":"<p>Created a mocked CAVEclient function for testing using the responses library to mock the server responses. This function returns a drop-in replacement for the <code>CAVEclient</code> function that will be able to initialize itself and selected individual service clients with the selected options.</p> <p>Note that the test configuration is intended to be purely for pytest purposes and should not actually result in calls to active endpoints.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>Name of the test datastack, by default None</p> <code>None</code> <code>global_server</code> <code>str</code> <p>Test global server address, by default None</p> <code>None</code> <code>local_server</code> <code>str</code> <p>Test local server address, by default None</p> <code>None</code> <code>info_file</code> <code>dictionary</code> <p>Info service dictionary for the datastack, by default None</p> <code>None</code> <code>chunkedgraph</code> <code>bool</code> <p>If True, configures the client to initialize a chunkedgraph subclient, by default False</p> <code>False</code> <code>chunkedgraph_server_version</code> <code>str</code> <p>Sets the value of the chunkedgraph server version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_CHUNKEDGRAPH_SERVER_VERSION.</p> <code>DEFAULT_CHUNKEDGRAPH_SERVER_VERSION</code> <code>chunkedgraph_api_versions</code> <code>list</code> <p>List of chunkedgraph API versions that the chunkedgraph client thinks exists, by default None. If None, returns the value in CHUNKEDGRAPH_API_VERSIONS.</p> <code>None</code> <code>materialization</code> <code>bool</code> <p>If True, configures the client to initalize a materialization subclient, by default False Note that materialization being set to True will also configure the chunkedgraph client.</p> <code>False</code> <code>materialization_server_version</code> <code>str</code> <p>Sets the value of the materialization server version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_MATERIALIZATION_SERVER_VERSON.</p> <code>DEFAULT_MATERIALIZATION_SERVER_VERSON</code> <code>available_materialization_versions</code> <code>list</code> <p>List of materialization database versions that the materialization client thinks exists, by default None. If None, returns the value in DEFAULT_MATERIALIZATION_VERSONS.</p> <code>None</code> <code>materialization_api_versions</code> <code>list</code> <p>List of materialization API versions that the materialization client thinks exists, by default None. If None, returns the value in MATERIALIZATION_API_VERSIONS.</p> <code>None</code> <code>set_version</code> <code>Optional[int]</code> <p>If set, will set the version of the materialization server to the value of set_version, by default None. To work, this version must be in the list of available materialization versions.</p> <code>None</code> <code>set_version_metadata</code> <code>Optional[dict]</code> <p>If set, will set the version metadata of the materialization server to the value of set_version_metadata. Default value is in DEFAULT_MATERIALIZATION_VERSION_METADATA.</p> <code>None</code> <code>json_service</code> <code>bool</code> <p>If True, configures the client to initalize a materialization subclient, by default False</p> <code>False</code> <code>json_service_server_version</code> <code>_type_</code> <p>Sets the value of the json state server version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_JSON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_JSON_SERVICE_SERVER_VERSION</code> <code>skeleton_service</code> <code>bool</code> <p>If True, configures the client to initalize a skeleton service subclient, by default False</p> <code>False</code> <code>skeleton_service_server_version</code> <code>_type_</code> <p>Sets the value of the skeleton service version as a three-element semanatic version (e.g \"2.3.4\"), by default the value in DEFAULT_SKELETON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_SKELETON_SERVICE_SERVER_VERSION</code> <code>l2cache</code> <code>bool</code> <p>If True, configures the client to initialize an l2cache subclient, by default False Note that l2cache being set to True will also configure the chunkedgraph client.</p> <code>False</code> <code>l2cache_disabled</code> <code>bool</code> <p>If True, allows a subclient to be initialized, but emulates a situation without an L2 cache, by default False Only used if l2cache is True.</p> <code>False</code> <code>global_only</code> <code>bool</code> <p>If True, only initializes the global services and does not use a datastack, by default False.</p> <code>False</code> <p>Returns:</p> Type Description <code>CAVEclient</code> <p>A mocked and initialized CAVEclient object for testing</p> <p>Examples:</p> <p>To make a basic pytest fixture to test chunkedgraph features with an initialized CAVEclient object in your pytest conftest.py file:</p> <pre><code>import pytest\nfrom caveclient.tools.testing import CAVEclientMock\n\ntest_datastack = \"test_stack\"\ntest_global_server = \"https://test.cave.com\"\ntest_local_server = \"https://local.cave.com\"\n\n@pytest.fixture()\ndef test_client():\n    return CAVEclientMock(\n        datastack_name=test_datastack,\n        global_server=test_global_server,\n        local_server=test_local_server,\n        chunkedgraph=True,\n    )\n\nYou can also create more complex fixtures with multiple services initialized and specific server versions:\n\n```python\n@pytest.fixture()\ndef fancier_test_client():\n    return CAVEclientMock(\n        datastack_name=test_datastack,\n        global_server=test_global_server,\n        local_server=test_local_server,\n        chunkedgraph=True,\n        chunkedgraph_server_version=\"3.0.2\",\n        materialization=True,\n        materialization_server_version=\"4.21.4\",\n        l2cache=True,\n    )\n</code></pre>"},{"location":"api/testing/#caveclient.tools.testing.get_materialiation_info","title":"<code>get_materialiation_info(materialization_versions=DEFAULT_MATERIALIZATION_VERSONS, version_metadata=DEFAULT_MATERIALIZATION_VERSION_METADATA)</code>","text":"<p>Get the materialization versions and version metadata for the materialization service.</p> <p>Parameters:</p> Name Type Description Default <code>materialization_versions</code> <code>list</code> <p>List of materialization database versions that the materialization client thinks exists, by default DEFAULT_MATERIALIZATION_VERSONS.</p> <code>DEFAULT_MATERIALIZATION_VERSONS</code> <code>version_metadata</code> <code>dict</code> <p>Version metadata for the materialization service, by default DEFAULT_MATERIALIZATION_VERSION_METADATA.</p> <code>DEFAULT_MATERIALIZATION_VERSION_METADATA</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"materialization_versions\", \"version_metadata\".</p>"},{"location":"api/testing/#caveclient.tools.testing.get_server_versions","title":"<code>get_server_versions(chunkedgraph_version=DEFAULT_CHUNKEDGRAPH_SERVER_VERSION, materialization_version=DEFAULT_MATERIALIZATION_SERVER_VERSON, skeleton_service_version=DEFAULT_SKELETON_SERVICE_SERVER_VERSION, json_service_version=DEFAULT_JSON_SERVICE_SERVER_VERSION)</code>","text":"<p>Get the server versions for the services used in testing.</p> <p>Parameters:</p> Name Type Description Default <code>chunkedgraph_version</code> <code>str</code> <p>Version of the chunkedgraph server, by default DEFAULT_CHUNKEDGRAPH_SERVER_VERSION.</p> <code>DEFAULT_CHUNKEDGRAPH_SERVER_VERSION</code> <code>materialization_version</code> <code>str</code> <p>Version of the materialization server, by default DEFAULT_MATERIALIZATION_SERVER_VERSON.</p> <code>DEFAULT_MATERIALIZATION_SERVER_VERSON</code> <code>skeleton_service_version</code> <code>str</code> <p>Version of the skeleton service server, by default DEFAULT_SKELETON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_SKELETON_SERVICE_SERVER_VERSION</code> <code>json_service_version</code> <code>str</code> <p>Version of the json service server, by default DEFAULT_JSON_SERVICE_SERVER_VERSION.</p> <code>DEFAULT_JSON_SERVICE_SERVER_VERSION</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"chunkedgraph_version\", \"materialization_version\", \"skeleton_service_version\", \"json_service_version\". Values are Version objects from packaging.versions.</p>"},{"location":"api/testing/#caveclient.tools.testing.get_server_information","title":"<code>get_server_information(datastack_name=TEST_DATASTACK, global_server=TEST_GLOBAL_SERVER, local_server=TEST_LOCAL_SERVER)</code>","text":"<p>Generate the datastack name and server locations used in testing.</p> <p>Parameters:</p> Name Type Description Default <code>datastack_name</code> <code>str</code> <p>Datastack value, by default the value in TEST_DATASTACK.</p> <code>TEST_DATASTACK</code> <code>global_server</code> <code>str</code> <p>Server for global services, by default TEST_GLOBAL_SERVER.</p> <code>TEST_GLOBAL_SERVER</code> <code>local_server</code> <code>str</code> <p>Server for local services, by default TEST_LOCAL_SERVER.</p> <code>TEST_LOCAL_SERVER</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"datastack_name\", \"local_server\", \"global_server\".</p>"},{"location":"api/testing/#caveclient.tools.testing.get_api_versions","title":"<code>get_api_versions(chunkedgraph_api_versions=CHUNKEDGRAPH_API_VERSIONS, materialization_api_versions=MATERIALIZATION_API_VERSIONS, schema_api_versions=SCHEMA_API_VERSIONS)</code>","text":"<p>Get the API versions for the services used in testing.</p> <p>Parameters:</p> Name Type Description Default <code>chunkedgraph_api_versions</code> <code>list</code> <p>List of chunkedgraph API versions that the chunkedgraph client thinks exists, by default CHUNKEDGRAPH_API_VERSIONS.</p> <code>CHUNKEDGRAPH_API_VERSIONS</code> <code>materialization_api_versions</code> <code>list</code> <p>List of materialization API versions that the materialization client thinks exists, by default MATERIALIZATION_API_VERSIONS.</p> <code>MATERIALIZATION_API_VERSIONS</code> <code>schema_api_versions</code> <code>list</code> <p>List of schema API versions that the schema client thinks exists, by default SCHEMA_API_VERSIONS.</p> <code>SCHEMA_API_VERSIONS</code> <p>Returns:</p> Type Description <code>dict</code> <p>Dictionary with keys: \"chunkedgraph_api_versions\", \"materialization_api_versions\", \"schema_api_versions\".</p>"},{"location":"api/testing/#caveclient.tools.testing.default_info","title":"<code>default_info(local_server=TEST_LOCAL_SERVER)</code>","text":"<p>Generate a info service info file for testing</p> <p>Parameters:</p> Name Type Description Default <code>local_server</code> <code>str</code> <p>Name of the local service, by default the value in TEST_LOCAL_SERVER.</p> <code>TEST_LOCAL_SERVER</code> <p>Returns:</p> Type Description <code>dict</code> <p>Info file for the datastack.</p>"},{"location":"tutorials/","title":"Getting Started","text":"<p>AnnotationFramework client is a package for simplifying interactions with HTML services associated with the CAVE (Connectome Annotation Versioning Engine).</p> <p>For a larger introduction to CAVE and its services, see the main GitHub organization site: https://github.com/CAVEconnectome</p>"},{"location":"tutorials/#installation","title":"Installation","text":"<p>The CAVEclient can be installed with pip:</p> <pre><code>$ pip install caveclient\n</code></pre>"},{"location":"tutorials/#assumptions","title":"Assumptions","text":"<p>The code is setup to work flexibly with any deployment of these services, but you need to specify the server_address if that address is not https://globalv1.daf-apis.com/ for each client when initializing it. Similarly, the clients can query the info service for metadata to simplify the interaction with a datastack, but you have to specify a datastack name.</p>"},{"location":"tutorials/advanced/","title":"Advanced Usage","text":""},{"location":"tutorials/advanced/#changing-session-configurations","title":"Changing session configurations","text":"<p>It is possible to change details of how a client talks to the various servers it needs to interface with. For instance, the <code>CAVEclient</code> will attempt to retry specific failed requests to the server, but will only try a specific number of times, and will only wait specific amounts of time between retries. These values can be changed via the <code>set_session_defaults</code> method. For instance, to change the number of retries to 5, and to increase the delay between subsequent retries, you could do:</p> <pre><code>from caveclient import set_session_defaults\n\nset_session_defaults(max_retries=5, backoff_factor=0.5)\n</code></pre> <p>Note that this needs to happen before initializing the client for this to work properly. Some of these parameters are also adjustable at the client level.</p> <p>To view the current session defaults, you can use the <code>get_session_defaults</code> method:</p> <pre><code>from caveclient import get_session_defaults\n\nclient.get_session_defaults()\n</code></pre> <p>More information on the available parameters can be found in the API documentation.</p>"},{"location":"tutorials/advanced/#writing-unit-tests-using-caveclient","title":"Writing unit tests using caveclient","text":"<p>If you have a module that relies on caveclient and you want to write unit tests for it, you can use the <code>caveclient.testing</code> module to help you create an initialized, mocked client with user-specified options. This helps deal with the complex conversation that caveclient has with various services when initializing itself so that you get the appropriate functionality. The function <code>CAVEclientMock</code> offers a number of options for how to build a client, including server versions for various services, available API versions, and more. Importantly, the module provides sensible defaults but also allows you to override them as needed. Note that using testing requires the responses to be installed, which is not a dependency of caveclient.</p> <p>The <code>CAVEclientMock</code> function requires you to specify which services you expect to call on in order to ensure that your code is working as expected. For example, to create a pytest fixture that would utilize the <code>chunkedgraph</code> module, but otherwise use default nonsense test values, you could include this in your <code>conftest.py</code>:</p> <pre><code>import pytest\nfrom caveclient.tools.testing import CAVEclientMock\n\n@pytest.fixture()\ndef test_client():\n    return CAVEclientMock(\n        chunkedgraph=True,\n    )\n</code></pre> <p>Then, in your test module, you can use the <code>test_client</code> fixture to get a client. Note that after mocked initialization, the client will attempt normal network requests and thus you should mock the responses. If you only care to get a specific value back from a given function, you can use pytest-mock to mock the response to a given function call. For example, if you have a function <code>do_something_with_roots</code> that takes a caveclient that uses the <code>get_roots</code> function, you could mock the <code>get_roots</code> function so that it return a specific value:</p> <pre><code>from pytest_mock import mocker\nfrom conftest import test_client\nfrom my_module import do_something_with_roots\n\ndef test_get_roots(mocker, test_client):\n    mocker.patch.object(test_client.chunkedgraph, 'get_roots', return_value=[1, 2, 3])\n    test_data = # Something appropriate\n    test_output = do_something_with_roots(test_client, test_data)\n    assert test_output == # The expected answer\n</code></pre> <p>Note that if you your own datastack <code>info_file</code> that has a different <code>local_server</code> address than the default value (<code>TEST_LOCAL_SERVER</code> variable, which defaults to <code>https://local.cave.com</code>) you will also need to specify a <code>local_server</code> with the same value as in the <code>local_server</code> field of your info file.</p>"},{"location":"tutorials/advanced/#specifying-conditions","title":"Specifying conditions","text":"<p>While sensible defaults are provided, you can also specify things like server versions to make sure your code works with the versions of the services you expect. For example, let's make a richer mock client that specifies the server versions for the <code>chunkedgraph</code>, <code>materailization</code>, and <code>l2cache</code> services:</p> <pre><code>@pytest.fixture()\ndef version_specified_client():\n    return CAVEclientMock(\n        chunkedgraph=True,\n        chunkedgraph_server_version='3.0.1',\n        materialization=True,\n        materialization_server_version='2.0.0',\n        l2cache=True,\n    )\n</code></pre> <p>Note that some services like <code>l2cache</code> do not currently use a server version to offer different functionality, and this value is not exposed for them currently. See the API documentation for more information.</p> <p>You can also override default values like <code>server_address</code> or <code>datastack_name</code>: <pre><code>@pytest.fixture()\ndef server_specified_client():\n    return CAVEclientMock(\n        datastack_name='my_datastack',\n        server_address='http://my.server.com',\n        chunkedgraph=True,\n        materialization=True,\n    )\n</code></pre></p> <p>If you want to get access to the various default values of server version, datastack name, datastack info file, and api versions, you can use functions <code>get_server_versions</code>, <code>get_server_information</code>, <code>default_info</code>, <code>get_api_versions</code> respectively. Each of these functions will return a dictionary of the values that can be used as a kwargs input into CAVEclientMock. If you specify your own override values, it will take precedence over the default value and you can just use the dictionary in your tests. See the caveclient tests for an example of how to use these functions.</p>"},{"location":"tutorials/annotation/","title":"Annotation","text":"<p>The AnnotationClient is used to interact with the AnnotationEngine service to create tables from existing schema, upload new data, and download existing annotations. Note that annotations in the AnnotationEngine are not linked to any particular segmentation, and thus do not include any root ids. An annotation client is accessed with <code>client.annotation</code>.</p>"},{"location":"tutorials/annotation/#getting-existing-tables","title":"Getting existing tables","text":"<p>A list of the existing tables for the datastack can be found with get_tables().</p> <pre><code>all_tables = client.annotation.get_tables()\nall_tables[0]\n</code></pre> <p>Each table has three main properties that can be useful to know:</p> <ul> <li><code>table_name</code> : The table name, used to refer to it when uploading or   downloading annotations. This is also passed through to the table in   the Materialized database.</li> <li><code>schema_name</code> : The name of the table's schema from   EMAnnotationSchemas (see below).</li> <li><code>max_annotation_id</code> : An upper limit on the number of annotations   already contained in the table.</li> </ul>"},{"location":"tutorials/annotation/#downloading-annotations","title":"Downloading annotations","text":"<p>You can download the JSON representation of a data point through the get_annotation() method. This can be useful if you need to look up information on unmaterialized data, or to see what a properly templated annotation looks like.</p> <pre><code>table_name = all_tables[0]['table_name']      # 'ais_analysis_soma'\nannotation_id = 100\nclient.annotation.get_annotation(annotation_ids=annotation_id, table_name=table_name)\n</code></pre>"},{"location":"tutorials/annotation/#create-a-new-table","title":"Create a new table","text":"<p>One can create a new table with a specified schema with the create_table() method:</p> <pre><code>client.annotation.create_table(table_name='test_table',\n                               schema_name='microns_func_coreg',\n                               voxel_resolution = [1,1,1],\n                               description=\"some text to describe your table\")\n</code></pre> <p>The voxel resolution is the units your position columns will be uploaded in [1,1,1] would imply a nm location, where as [4,4,40] would correspond to voxels of that size. If you are uploading points from a neuroglancer session, you want this to match the units of that neuroglancer view.</p> <p>Note there are some optional metadata parameters to create_table()</p> <ul> <li><code>notice_text</code> : This is text that will show up to users who access   this data as a warning. This could be used to warn users that the   data is not complete or checked yet, or to advertise that a   particular publication should be cited when using this table.</li> <li><code>read_permission</code> : one of \"PRIVATE\" which means only you can read   data in this table. \"PUBLIC\" (default) which means anyone can read   this table that has read permissions to this dataset. So if and only   if you can read the segmentation results of this data, you can read   this table. \"GROUP\" which means that you must share a common group   with this user for them to be able to read. We need to make a way to   discover what groups you are in and who you share groups with.</li> <li><code>write_permission</code>: one of \"PRIVATE\" (default), which means only   you can write to this table. \"PUBLIC\" which means anyone can write   to this table that has write permissions to this dataset. Note   although this means anyone can add data, no annotations are ever   truly overwritten. \"GROUP\" which means that you must share a   common group with this user for them to be able to write. We need to   make a way to discover what groups you are in and who you share   groups with.</li> </ul> <p>If you change your mind about what you want for metadata, some but not all fields can be updated with update_metadata(). This includes the description, the notice_text, and the permissions, but not the name, schema or voxel resolution.</p> <pre><code># to update description\nclient.annotation.update_metadata(table_name='test_table',\n                                  description=\"a new description for my table\")\n\n# to make your table readable by anybody who can read this dataset\nclient.annotation.update_metadata(table_name='test_table',\n                                  notice_text=\"This table isn't done yet, don't trust it. Contact me\")\n\n# to make your table readable by anybody who can read this dataset\nclient.annotation.update_metadata(table_name='test_table',\n                                  read_permisison=\"PUBLIC\")\n</code></pre> <p>New data can be generated as a dict or list of dicts following the schema and uploaded with <code>post_annotation</code>. For example, a <code>microns_func_coreg</code> point needs to have:</p> <ul> <li><code>type</code> set to <code>microns_func_coreg</code></li> <li><code>pt</code> set to a dict with <code>position</code> as a key and   the xyz location as a value.</li> <li><code>func_id</code> set to an integer.</li> </ul> <p>The following could would create a new annotation and then upload it to the service. Note that you get back the annotation id(s) of what you uploaded.</p> <pre><code>new_data = {'type': 'microns_func_coreg',\n            'pt': {'position': [1,2,3]},\n            'func_id': 0}\nclient.annotation.post_annotation(table_name='test_table', data=[new_data])\n</code></pre> <p>There are methods to simplify annotation uploads if you have a pandas dataframe whose structure mirrors the struction of the annotation schema you want to upload</p> <pre><code>import pandas as pd\n\ndf = pd.DataFrame([{'id':0,\n         'type': 'microns_func_coreg',\n         'pt_position': [1,2,3]},\n         'func_id': 0},\n        {'id':1,\n        'type': 'microns_func_coreg',\n        'pt_position': [3,2,1]},\n        'func_id': 2}])\nclient.annotation.post_annotation_df('test_table', df)\n</code></pre> <p>Note that here I specified the IDs of my annotations, which you can do, but then its up to you to assure that the IDs don't collide with other IDs. If you leave them blank then the service will assign the IDs for you.</p> <p>There is a similar method for updating update_annotation_df()</p>"},{"location":"tutorials/annotation/#staged-annotations","title":"Staged Annotations","text":"<p>Staged annotations help ensure that the annotations you post follow the appropriate schema, both by providing guides to the field names and locally validating against a schema before uploading. The most common use case for staged annotations is to create a StagedAnnotation object for a given table, then add annotations to it individually or as a group, and finally upload to the annotation table.</p> <p>To get a StagedAnnotation object, you can start with either a table name or a schema name. Here, we'll assume that there's already a table called \"my_table\" that is running a \"cell_type_local\" schema. If we want to add new annotations to the table, we simply use the table name with stage_annotations().</p> <pre><code>stage = client.annotation.stage_annotations(\"my_table\")\n</code></pre> <p>This <code>stage</code> object retrieves the schema for the table and hosts a local collection of annotations. Every time you add an annotation, it is immediately validated against the schema. To add an annotation, use the <code>add</code> method:</p> <pre><code>stage.add(\n    cell_type = \"pyramidal_cell\",\n    classification_system=\"excitatory\",\n    pt_position=[100,100,10],\n)\n</code></pre> <p>The argument names derive from fields in the schema and you must provide all required fields. Any number of annotations can be added to the stage. A dataframe of annotations can also be added with <code>stage.add_dataframe</code>, and requires an exact match between column names and schema fields. The key difference between this and posting a dataframe directly is that annotations added to a StagedAnnotations are validated locally, allowing any issues to be caught before uploading.</p> <p>You can see the annotations as a list of dictionary records with <code>stage.annotation_list</code> or as a Pandas dataframe with <code>stage.annotation_dataframe</code>. Finally, if you initialized the stage with a table name, this information is stored in the <code>stage</code> and you can simply upload it from the client.</p> <pre><code>client.annotation.upload_staged_annotations(stage)\n</code></pre> <p>Updating annotations requires knowing the annotation id of the annotation you are updating, which is not required in the schema otherwise. In order to stage updated annotations, set the <code>update</code> parameter to <code>True</code> when creating the stage.</p> <pre><code>update_stage = client.annotation.stage_annotations(\"my_table\", update=True)\nupdate_stage.add(\n    id=1,\n    cell_type = \"stellate_cell\",\n    classification_system=\"excitatory\",\n    pt_position=[100,100,10],\n)\n</code></pre> <p>The <code>update</code> also informs the framework client to treat the annotations as an update and it will use the appropriate methods automatically when uploading <code>client.annotation.upload_staged_annotations</code>.</p> <p>If you want to specify ids when posting new annotations, <code>id_field</code> can be set to True when creating the StagedAnnotation object. This will enforce an <code>id</code> column but still post the data as new annotations.</p> <p>If you might be adding spatial data in coordinates that might be different than the resolution for the table, you can also set the <code>annotation_resolution</code> when creating the stage. The stage will convert between the resolution you specify for your own annotations and the resolution that the table expects.</p> <pre><code>stage = client.annotation.stage_annotations(\"my_table\", annotation_resolution=[8,8,40])\nstage.add(\n    cell_type='pyramidal_cell',\n    classification_system=\"excitatory\",\n    pt_position=[50,50,10],\n)\n</code></pre>"},{"location":"tutorials/authentication/","title":"Authentication","text":"<p>Authentication tokens are generally needed for programmatic access to our services. The AuthClient handles storing and loading your token or tokens and inserting it into requests in other clients.</p> <p>We can access the auth client from <code>client.auth</code>. Once you have saved a token, you probably won't interact with this client very often, however it has some convenient features for saving new tokens the first time. Let's see if you have a token already. Probably not.</p> <pre><code>client = CAVEclient()\nauth = client.auth\nprint(f\"My current token is: {auth.token}\")\n</code></pre>"},{"location":"tutorials/authentication/#getting-a-new-token","title":"Getting a new token","text":"<p>To get a new token, you will need to manually acquire it. For convenience, the function client.auth.get_new_token() provides instructions for how to get and save the token.</p> <p>By default, the token is saved to <code>~/.cloudvolume/secrets/cave-secret.json</code> as a string under the key <code>token</code>. This makes it compatible by default with Cloudvolume projects, which can come in handy. The following steps will save a token to the default location.</p> <pre><code>auth.get_new_token()\n</code></pre> <pre><code>new_token = 'abcdef1234567890' #This is the text you see after you visit the website.\nauth.save_token(token=new_token)\nprint(f\"My token is now: {auth.token}\")\n</code></pre> <p>Note that requesting a new token will invalidate your previous token on the same project. If you want to use the same token across different computers, you will need to share the same token information.</p>"},{"location":"tutorials/authentication/#loading-saved-tokens","title":"Loading saved tokens","text":"<p>Try opening <code>~/.cloudvolume/secrets/cave-secret.json</code> to see what we just created.</p> <p>If we had wanted to use a different file or a different json key, we could have specified that in auth.save_token.</p> <p>Because we used the default values, this token is used automatically when we initialize a new CAVEclient. If we wanted to use a different token file, token key, or even directly specify a token we could do so here.</p> <pre><code>client = CAVEclient(datastack_name)\nprint(f\"Now my basic token is: {client.auth.token}\")\n\nclient_direct = CAVEclient(datastack_name, auth_token='another_fake_token_678')\nprint(f\"A directly specified token is: {client_direct.auth.token}\")\n</code></pre> <p>If you use a CAVEclient, the AuthClient and its token will be automatically applied to any other services without further use.</p>"},{"location":"tutorials/chunkedgraph/","title":"Chunked Graph","text":"<p>The chunkedgraph is a dynamic oct-tree connected components supervoxel graph.</p> <p></p> <p>A visual representation of an oct-tree (Wikipedia (WhiteTimberwolf) CC BY-SA 3.0)</p> <p>As with any oct-tree, it is organized in hierarchical levels, with the bottom level 1 corresponding to the supervoxels of the segmentations, and the top level being the unique connected components of the supervoxel graph.</p> <p></p> <p>A figure illustrating the spatial chunking and editing of a the chunkedgraph. From Dorkenwald et. al 2021</p> <p>The ChunkedGraph client allows one to interact with the ChunkedGraph service, which stores and updates the supervoxel agglomeration graph. This is most often useful for looking up an object root id of a supervoxel or looking up supervoxels belonging to a root id. The ChunkedGraph client is at <code>client.chunkedgraph</code>.</p>"},{"location":"tutorials/chunkedgraph/#look-up-a-supervoxel","title":"Look up a supervoxel","text":"<p>Usually in Neuroglancer, one never notices supervoxel ids, but they are important for programmatic work. In order to look up the root id for a location in space, one needs to use the supervoxel segmentation to get the associated supervoxel id. The ChunkedGraph client makes this easy using the client.chunkedgraph.get_root_id() method.</p> <pre><code>sv_id = 104200755619042523\nclient.chunkedgraph.get_root_id(supervoxel_id=sv_id)\n</code></pre> <p>However, as proofreading occurs, the root id that a supervoxel belongs to can change. By default, this function returns the current state, however one can also provide a UTC timestamp to get the root id at a particular moment in history. This can be useful for reproducible analysis. Note below that the root id for the same supervoxel is different than it is now.</p> <pre><code>import datetime\n\n# I looked up the UTC POSIX timestamp from a day in early 2019.\ntimestamp = datetime.datetime.utcfromtimestamp(1546595253)\n\nsv_id = 104200755619042523\nclient.chunkedgraph.get_root_id(supervoxel_id=sv_id, timestamp=timestamp)\n</code></pre> <p>If you are doing this across lots of supervoxels (or any nodes) then you can do it more efficiently in one request with client.chunkedgraph.get_roots()</p> <pre><code>node_ids = [104200755619042523, 104200755619042524,104200755619042525]\nroot_ids = client.chunkedgraph.get_roots(node_ids)\n</code></pre>"},{"location":"tutorials/chunkedgraph/#getting-supervoxels-for-a-root-id","title":"Getting supervoxels for a root id","text":"<p>A root id is associated with a particular agglomeration of supervoxels, which can be found with the client.chunkedgraph.get_leaves() method. A new root id is generated for every new change in the chunkedgraph, so time stamps do not apply.</p> <pre><code>root_id = 648518346349541252\nclient.chunkedgraph.get_leaves(root_id)\n</code></pre> <p>You can also query the chunkedgraph not all the way to the bottom, using the stop_layer option</p> <pre><code>root_id = 648518346349541252\nclient.chunkedgraph.get_leaves(root_id,stop_layer=2)\n</code></pre> <p>This will get all the level 2 IDs for this root, which correspond to the lowest chunk of the hierarchy. An analogous option exists for client.chunkedgraph.get_roots(). This is useful to help find nodes to query within the <code>l2cache</code>, amongst other things.</p>"},{"location":"tutorials/chunkedgraph/#other-functions","title":"Other functions","text":"<p>There are a variety of other interesting functions to explore in client.chunkedgraph.</p>"},{"location":"tutorials/framework/","title":"Introduction","text":""},{"location":"tutorials/framework/#caveclient-one-client-for-all-services","title":"CAVEclient: one client for all services","text":"<p>The CAVE Framework consists of a number of different services, each with a specific set of tasks that it can perform through REST endpoints. The CAVEclient is designed to ease programmatic interaction with all of the various endpoints. In addition, most programmatic access requires the use of authentication tokens. In order to collect a given server, datastack name, and user token together into a coherent package that can be used on multiple endpoints, the CAVEclient builds appropriately configured clients for each of the specific services. Each of the individual services has their own specific documentation as well.</p>"},{"location":"tutorials/framework/#global-and-local-services","title":"Global and Local Services","text":"<p>There are two categories of data in CAVE: Global and local. Local services are associated with a single so-called datastack, which refers to a precise collection of imagery and segmentation data that function together. For example, EM imagery and a specific pychunkedgraph segmentation would be one datastack, while the same EM imagery but an initial static segmentation would be another. Datastacks are referred to by a short name, for instance <code>pinky100_public_flat_v185</code>.</p> <p>Global services are those that are potentially shared across multiple different specific datastacks. These include the info service, which can describe the properties of all available datastacks, the authentication service, and the state service that hosts neuroglancer states. Global services are associated with a particular URL (by default <code>http://globalv1.daf-apis.com</code>), but not a single datastack.</p>"},{"location":"tutorials/framework/#initializing-a-caveclient","title":"Initializing a CAVEclient","text":"<p>Assuming that the services are on <code>http://globalv1.daf-apis.com</code> and authentication tokens are either not being used or set up with default values (see Authentication), a simple CAVEclient that can only access global services can be initialized:</p> <pre><code>from caveclient import CAVEclient\n\nclient = CAVEclient()\n</code></pre> <p>Just to confirm that this works, let's see if we can get the EM image source from the InfoService. If you get a list of names of datastacks, all is good. If you have not yet set up an authentication token or you get an authentication error, look at Getting a new token for information about how to set up your auth token.</p> <pre><code>client.info.get_datastacks()\n</code></pre> <p>If you have a specific datastack you want to use, you can initialize your CAVEclient with it. This gives you access to the full range of client functions.</p> <pre><code>client = CAVEclient(datastack_name='my_datastack')\n</code></pre>"},{"location":"tutorials/framework/#using-other-server-addresses","title":"Using Other Server Addresses","text":"<p>If your data is hosted by a different global server, you specify its address when initializing the client.</p> <pre><code>client = CAVEclient(datastack_name='my_datastack', server_address='http://global.myserver.com')\n</code></pre> <p>By default, if you pass both a server address and a datastack, the client will store the mapping from datastack to server address in the same location as the default for authentication tokens. Once stored, the client will automatically use the correct server address for the datastack if none is provided. You can override storing the server address by passing <code>write_server_address=False</code>. Datastacks can be removed from the cache using</p> <p>caveclient.datastack_lookup.reset_server_address_cache(datastack_name).</p>"},{"location":"tutorials/framework/#accessing-specific-clients","title":"Accessing specific clients","text":"<p>Each client can be accessed as a property of the main client. See the documentation at left for the capabilities of each. Assuming your client is named <code>client</code>, the subclients for each service are:</p> <ul> <li>Authentication Service : <code>client.auth</code></li> <li>AnnotationEngine : <code>client.annotation</code></li> <li>PyChunkedGraph : <code>client.chunkedgraph</code></li> <li>InfoService : <code>client.info</code></li> <li>EM Annotation Schemas : <code>client.schemas</code></li> <li>JSON Neuroglancer State Service : <code>client.state</code></li> <li>Skeleton Service : <code>client.skeletonservice</code></li> </ul>"},{"location":"tutorials/info/","title":"Info Service","text":"<p>A datastack has a number of complex paths to various data sources that together comprise a datastack. Rather than hardcode these paths, the InfoService allows one to query the location of each data source. This is also convenient in case data sources change.</p> <p>An InfoClient is accessed at <code>client.info</code>.</p> <pre><code>client = CAVEclient(datastack_name)\nprint(f\"This is an info client for {client.info.datastack_name} on {client.info.server_address}\")\n</code></pre>"},{"location":"tutorials/info/#accessing-datastack-information","title":"Accessing datastack information","text":"<p>All of the information accessible for the datastack can be seen as a dict using <code>get_datastack_info()</code>.</p> <pre><code>info.get_datastack_info()\n</code></pre> <p>Individual entries can be found as well. Use tab autocomplete to see the various possibilities.</p> <pre><code>info.graphene_source()\n</code></pre>"},{"location":"tutorials/info/#adjusting-formatting","title":"Adjusting formatting","text":"<p>Because of the way neuroglancer looks up data versus cloudvolume, sometimes one needs to convert between <code>gs://</code> style paths to <code>https://storage.googleapis.com/</code> stype paths. All of the path sources in the info client accept a <code>format_for</code> argument that can handle this, and correctly adapts to graphene vs precomputed data sources.</p> <pre><code>neuroglancer_style_source = info.image_source(format_for='neuroglancer')\nprint(f\"With gs-style: { neuroglancer_style_source }\")\n\ncloudvolume_style_source = info.image_source(format_for='cloudvolume')\nprint(f\"With https-style: { cloudvolume_style_source }\")\n</code></pre>"},{"location":"tutorials/l2cache/","title":"Level 2 Cache","text":"<p>To understand the level 2 cache, you must understand the structure of the chunkedgraph so see the chunkedgraph tutorial.</p> <p>Nodes on the second level or layer of the graph, corresponds to all the supervoxels that are locally connected to one another within a single level 2 spatial \"chunk\" of the data. The Level 2 Cache, is a service whose job it is to track and update relevant statistics about every level 2 node within the a chunkedgraph. The source code of this service can be found here.</p>"},{"location":"tutorials/l2cache/#finding-level-2-nodes","title":"Finding Level 2 Nodes","text":"<p>The chunkedgraph can be used to find the level2 nodes of a rootID using a <code>stop_layer=2</code> keyword argument on the client.chunkedgraph.get_leaves(). Conversely the level 2 node of a supervoxel can be found using the same keyword argument of client.chunkedgraph.get_roots(). Note if you don't specify a timestamp it will give you the level2 node that is presently associated with the object.</p>"},{"location":"tutorials/l2cache/#statistics","title":"Statistics","text":"<p>The statistics that are available are:</p> <ul> <li>area_nm2: The surface area of the object in square nanometers. Does not include border touching voxels</li> <li>size_nm3: The volume of the object in cubic nanometers,   based on counting voxels in the object.</li> <li>max_dt_nm: The maximum edge distance transform of that   object in nanometers. Meant to capture the   maximum \"thickness\" of the voxels in the   node.</li> <li>mean_dt_nm: The average edge distance transform of that   object in nanometers. Meant to capture the   average \"thickness\" of voxels in that node.</li> <li>rep_coord_nm: A list of x,y,z coordinates in nanometers that   represent a point within the object that is   designed to be close to the \"center\" of the   object. This is the location of the max_dt_nm   value.</li> <li>chunk_intersect_count: A 2 x 3 matrix representing the 6 sides of the   chunk, and whose values represent how many   voxels border that side of the chunk. Meant to   help understand significant the borders with   other chunks are. Ordering is the [[x_bottom, y_bottom, z_bottom],[x_top, y_top, z_top]]   where {xyz}_bottom refers to the face which   has the smallest values for that dimension, and   {xyz}_top refers to the face which has the   largest.</li> <li>pca A 3x3 matrix representing the principal   components of the xyz point cloud of voxels for   this object. Ordering is NxD where N is the   components and D are the xyz dimensions. Meant   to help desribe the orientation of the level 2   chunk. Note that this is not calculated for   very small objects and so might not be present   for all level 2 nodes. You will see that its   availability correlates strongly with size_nm3.</li> <li>pca_val The 3 principal component values for the PCA   components.</li> </ul>"},{"location":"tutorials/l2cache/#retrieving-level-2-statistics","title":"Retrieving Level 2 Statistics","text":"<p>Level 2 stats about nodes can be retreived using the client.l2cache.get_l2data() method. It simply takes a list of level 2 nodes you want to retrieve. Optionally you can specify only the attributes that you are interested in retrieving which will speed up the request.</p>"},{"location":"tutorials/l2cache/#missing-data","title":"Missing Data","text":"<p>The service is constantly watching for changes made to objects and recalculating stats on new level2 nodes that are created, in order to keep its database of statistics current. This however takes some time, and is subject to sporadic rare failures. If you request stats on a level 2 node which are not in the database, you will receive an empty dictionary for that node. This will immediately trigger the system to recalculate the statistics of that missing data, and so it should be available shortly (on the order of seconds) if systems are operational. Please note that PCA is not calculated for very small objects because it is not meaningful. So if you are interested in differentiating whether PCA is not available because it hasn't been calculated, vs when its not available because it is not possible to calculate, you should ask for at least one other non PCA statistic as well. You will see that its availability correlates strongly with <code>size_nm3</code>.</p>"},{"location":"tutorials/l2cache/#use-cases","title":"Use Cases","text":""},{"location":"tutorials/l2cache/#calculate-total-area-and-volume-of-cells","title":"Calculate Total Area and Volume of Cells","text":"<p>Say you want to calculate the total surface area and volume of a object in the dataset. The areas and volume of each component can simply be added together to do this.</p> <pre><code>import pandas as pd\nroot_id = 648518346349541252\nlvl2nodes = client.chunkedgraph.get_leaves(root_id,stop_layer=2)\nl2stats = client.l2cache.get_l2data(lvl2nodes, attributes=['size_nm3','area_nm2'])\nl2df = pd.DataFrame(l2stats).T\ntotal_area_um2=l2df.area_nm2.sum()/(1000*1000)\ntotal_volume_um3 = l2df.size_nm3.sum()/(1000*1000*1000)\n</code></pre> <p>By utilizing the bounds argument of get_leaves, you can also do simple spatially restricted analysis of objects. In fact, because you have data on each level2 node individually, you can segregate the neuron using any labelling of its topology.</p>"},{"location":"tutorials/l2cache/#skeletonization","title":"Skeletonization","text":"<p>Level 2 nodes have \"cross chunk\" edges within the chunkedgraph which represent what level 2 nodes that object is locally connected to. This forms a graph between the level 2 nodes of the object that can be retrieved using the chunkedgraph function client.chunkedgraph. This graph represents a topological representation of the neuron at the resolution of individual chunks, and is guaranteed to be fully connected, unlike a voxel or mesh representation of the neuron which can have gaps where there are defects in the segmentation volume or incorrectly inferred edges at self contact locations.</p> <p>The level 2 graph can be turned into a skeleton representation of a neuron using a graph based TEASAR like algorithm as described for skeletonizing meshes in this MeshParty Documentation. There is an implementation of this approach that utilizes the chunkedgraph and the L2cache if available here and on pypi as <code>pcg-skel</code>. In this implementation the l2cache is used to more accurately place the level 2 nodes in space using the <code>rep_coord_nm</code> value.</p> <p>Note that there is detailed documentation on the Skeleton Client interface at the skeletonization tutorial.</p>"},{"location":"tutorials/l2cache/#trajectory-distributions","title":"Trajectory Distributions","text":"<p>If one is interested in the bulk direction of processes in a region of the brain, one can start with supervoxels in a region, find level 2 nodes that correspond to them, filter out components based on size, (or other criteria such as whether they are part of objects that have components in some other brain area) and look at the distribution of PCA components to understand the directions that those processes are moving within that region of space.</p>"},{"location":"tutorials/materialization/","title":"Materialization","text":"<p>The Materialization client allows one to interact with the materialized annotation tables, that were posted to the annotation service (the annotations tutorial).</p> <p>To see the entire class visit the API doc.</p> <p>The service regularly looks up all annotations and the segids underneath all the boundspatialpoints. You can then query these tables to find out the IDs that underlie the annotations, or the annotations that now intersect with certain IDs.</p> <p>For example, one common pattern is that you have identified a cell based on the location of its cell body, and you have an annotation there.</p> <p>You want to know what are the inputs onto the cell, so you first query the annotation table with your soma annotation, asking for the current ID underneath that soma. Then you query a synapse table for all synapse annotations that have a post-synaptic ID equal to the ID from your soma annotation.</p> <p>In this way your code stays the same, as the proofreading changes and you can track the connectivity of your cell over time.</p>"},{"location":"tutorials/materialization/#initializing-the-client","title":"Initializing the client","text":"<p>By default when you initialize the overall client, it will choose the most recent materialization version available. This may or may not be desirable depending on your use case. If your code involves using specific IDs then you should be using a specific version that is tied to a timepoint where those IDs are valid.</p> <p>To see what versions are available, use the client.materialize.get_versions() function.</p> <pre><code>client.materialize.get_versions()\n</code></pre> <p>Each version has a timestamp it was run on as well as a date when it will expire. You can query all this metadata for a specific version using client.materialize.get_version_metadata() or all versions using client.materialize.get_versions_metadata().</p> <p>To change the default version, alter the <code>.version</code> property of the client. This will change the version for all subsequent calls which expect one, unless you specify a different version in the method call. Note that this also sets the <code>timestamp</code> property of the client to the timestamp of the version for methods which expect a timestamp.</p> <pre><code>client.version = 9\n</code></pre> <p>You can also specify the version when you initialize the client, e.g.:</p> <pre><code>client = CAVEclient('minnie65_public', version=661)\n</code></pre> <p>Or, you can specify the version when making a particular method call.</p>"},{"location":"tutorials/materialization/#browsing-versions","title":"Browsing versions","text":"<p>To see what tables are available in a version you can use client.materialize.get_tables().</p> <p>If you want to read about the description of what that table is, use the annotationengine client client.materialize.get_table_metadata().</p> <p>If you want to read more about the schema for the annotation table use the schema service client.schema.schema_definition().</p> <p>Note, the materialization service has a human readable webpage that links to the other services that might be more convenient for you to browse, to get a link there in ipython display <code>client.materialize.homepage</code></p> <p>for some important tables, the info service has a pointer to which table you should use in the metadata for the datastack. <code>client.info.get_datastack_info()['synapse_table']</code> and <code>client.info.get_datastack_info()['soma_table']</code>.</p> <p>To see how many annotations are in a particular table use</p> <pre><code>nannotations=client.materialize.get_annotation_count('my_table')\n</code></pre>"},{"location":"tutorials/materialization/#querying-tables","title":"Querying tables","text":"<p>To query a small table, you can just download the whole thing using client.materialize.query_table() which will return a dataframe of the table.</p> <p>Note however, some tables, such as the synapse table might be very large 200-300 million rows and the service will only return the first 200,000 results, and not in a deterministic manner. NOTE! This API is not designed to enable enmass downloading of the entire synapse table there are more efficient ways of doing this. Contact your dataset administrator for more information if this is what you are looking to do.</p> <p>To just get a preview, use the limit argument (but note again that this won't be a reproducible set)</p> <pre><code>df=client.materialize.query_table('my_table', limit=10)\n</code></pre> <p>For most applications, you will want to filter the query in some way.</p> <p>We offer seven kinds of filters you can apply: <code>filter_in_dict</code> <code>filter_out_dict</code>, <code>filter_equal_dict</code>, <code>filter_greater_dict</code>, <code>filter_less_dict</code>, <code>filter_greater_equal_dict</code>, and <code>filter_less_equal_dict</code>. For query_table each is specified as a dictionary where the keys are column names, and the values are a list of values (or single value in the case of filter_equal).</p> <p>So for example to query a synapse table for all synapses onto a neuron in flywire you would use</p> <pre><code>synapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.query_table(synapse_table,\n                                  filter_equal_dict = {'post_pt_root_id': MYID})\n</code></pre> <p>The speed of querying is affected by a number of factors, including the size of the data. To improve the performance of results, you can reduce the number of columns returned using <code>select_columns</code>.</p> <p>So for example, if you are only interested in the root_ids and locations of pre_synaptic terminals you might limit the query with select_columns. Also, it is convenient to return the with positions as a column of <code>np.array([x,y,z])</code> coordinates for many purposes. However, sometimes you might prefer to have them split out as separate x, y, z columns. To enable this option use <code>split_columns=True</code>. split_columns=True is faster, as combining them is an extra step.</p> <pre><code>synapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.query_table(synapse_table,\n                                  filter_equal_dict = {'post_pt_root_id': MYID},\n                                  select_columns=['id','pre_pt_root_id', 'pre_pt_position'],\n                                  split_columns=True)\n</code></pre>"},{"location":"tutorials/materialization/#desired-resolution","title":"Desired Resolution","text":"<p>Often you want to have position information in different units. For example, to consider synapse locations or soma locations, you might want to have positions in nanometers or microns.</p> <p>To create neuroglancer views, you might want positions in integer voxels of a size that aligns with the resolution you are used to using Neuroglancer at.</p> <p>Annotation tables can be created and uploaded in varying resolutions according to whatever the user of the table felt was natural. This information is available in the metadata for that table. In addition, you may pass desired_resolution as a keyword argument which will automatically convert all spatial positions into voxels of that size in nanometers.</p> <p>So if you want positions in nanometers, you would pass <code>desired_resolution=[1,1,1]</code>. If you want positions in microns you would pass <code>desired_resolution=[1000,1000,1000]</code>. If you want positions in 4,4,40nm voxel coordinates to use with cloud-volume or neuroglancer you would pass <code>desired_resolution=[4,4,40]</code>.</p>"},{"location":"tutorials/materialization/#spatial-filters","title":"Spatial Filters","text":"<p>You can also filter columns that are associated with spatial locations based upon being within a 3d bounding box.</p> <p>This is done by adding a filter_spatial_dict argument to query_table. The units of the bounding box should be in the units of the voxel_resolution of the table (which can be obtained from client.materialize.get_table_metadata()).</p> <pre><code>bounding_box = [[min_x, min_y, min_z], [max_x, max_y, max_z]]\nsynapse_table = client.info.get_datastack_info('synapse_table')\ndf=client.materialize.query_table(synapse_table,\n                                  filter_equal_dict = {'post_pt_root_id': MYID},\n                                  filter_spatial_dict = {'post_pt_position': bounding_box})\n</code></pre>"},{"location":"tutorials/materialization/#synapse-query","title":"Synapse Query","text":"<p>For synapses in particular, we have a simplified method for querying them with a reduced syntax. client.materialize.synapse_query() lets you specify pre and post synaptic partners as keyword arguments and bounding boxes. The defaults make reasonable assumptions about what you want to query, namely that the synapse_table is the table that the info service advertises, and that if you specify a bounding box, that you want the post_pt_position. These can be overridden of course, but the above bounding box query is simplified to.</p> <p>NOTE! This API is not designed to enable enmass downloading of the entire synapse table there are more efficient ways of doing this. Contact your dataset administrator for more information if this is what you are looking to do.</p> <pre><code>bounding_box = [[min_x, min_y, min_z], [max_x, max_y, max_z]]\ndf=client.materialize.query_table(post_ids = MYID,\n                                  bounding_box=bounding_box)\n</code></pre>"},{"location":"tutorials/materialization/#live-query","title":"Live Query","text":"<p>In order to query the materialized tables above you can only use IDs that were present at the timestamp of the materialization. If you query the tables with an ID that is not valid during the time of the materialization you will get empty results.</p> <p>To check if root_ids are valid at your materialization's timestamp, you can use client.chunkedgraph.is_latest_roots()</p> <pre><code>import numpy as np\nmat_time = client.materialize.get_timestamp()\nis_latest = client.chunkedgraph.is_latest_roots([MYID], timestamp=mat_time)\nassert(np.all(is_latest))\n</code></pre> <p>If you need to lookup what happened to that ID, you can use the chunkedgraph lineage tree, to look into the future or the past, depending on your application you can use client.chunkedgraph.get_lineage_graph().</p> <p>Again, the ideal situation is that you have an annotation in the database which refers to your objects of interest, and querying that table by the id column will return the object in the most recent materialization.</p> <p>However, sometimes you might be browsing and proofreadding the data and get an ID that is more recent that the most recent version available. For convenience, you can use client.materialize.live_query().</p> <p>to automatically update the results of your query to a time in the future, such as now. For example, to pass now, use <code>datetime.datetime.now(datetime.timezone.utc)</code>. Note all timestamps are in UTC throughout the codebase.</p> <pre><code>import datetime\nsynapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.live_query(synapse_table,\n                                  datetime.datetime.now(datetime.timezone.utc),\n                                  filter_equal_dict = {'post_pt_root_id': MYID})\n</code></pre> <p>This will raise an ValueError exception if the IDs passed in your filters are not valid at the timestamp given</p> <p>You can also pass a timestamp directly to query_table and it will call live_query automatically.</p> <pre><code>import datetime\nsynapse_table = client.info.get_datastack_info()['synapse_table']\ndf=client.materialize.query_table(synapse_table,\n                                  timestamp=datetime.datetime.now(datetime.timezone.utc),\n                                  filter_equal_dict = {'post_pt_root_id': MYID})\n</code></pre> <p>Also, keep in mind if you run multiple queries and at each time pass <code>datetime.datetime.now(datetime.timezone.utc)</code>, there is no guarantee that the IDs will be consistent from query to query, as proofreading might be happening at any time. For larger scale analysis constraining oneself to a materialized version will ensure consistent results.</p> <p>Versions have varying expiration times in order to support the tradeoff between recency and consistency, so before undertaking an analysis project consider what version you want to query and what your plan will be to update your analysis to future versions.</p>"},{"location":"tutorials/materialization/#content-aware-interface-experimental","title":"Content-aware Interface (Experimental)","text":"<p>As of version 5.8.0, we have introduced a new interface to query tables and views. This interface might have small but breaking changes in the near future. :::</p> <p>In order to make the querying interface more consistent across tables, we have introduced an additional alternative interface to filtering and querying data via the <code>client.materialize.tables</code> object. When you instantiate this object, this object finds all of the existing tables and the list of their columns and lets you filter the tables as arguments in the function with suggestions. Moreover, the filtering arguments and the querying arguments are separated into two.</p> <p>Let's see how this works with a simplest example --- downloading a table called <code>nucleus_detection_v0</code>. First, we reference the table as a function and then we run the query --- this is exactly the same as <code>client.materialize.query_table('nucleus_detection_v0')</code>.</p> <pre><code>client = CAVEclient('minnie65_public')\nnuc_df = client.materialize.tables.nucleus_detection_v0().query()\n</code></pre> <p>Where things differ is when we add filters. If we want to query based on a set of values for the field \"id\", for example, we add that as an argument:</p> <pre><code>my_ids = [373879, 111162]\nnuc_df = client.materialize.tables.nucleus_detection_v0(id=my_ids).query()\n</code></pre> <p>Where in this example the <code>id=</code> queries the column <code>id</code> based on the schema. These values can be either individual elements (i.e. an integer or a string) or a list/array of elements, and any field can be used. The tooling will automatically sort out how to format the filtering appropriately when running the query. Importantly, the filtering is identical between querying all types of tables and queries. To see the complete list of fields that can be queried, you can tab-autocomplete or in Jupyter or IPython glance at the docstring with <code>client.materialize.tables.nucleus_detection_v0?</code>.</p> <p>In addition to filtering by one or many values, you can also do spatial queries. To find only annotations within a particular bounding box, you need to specify the spatial point (e.g. <code>pt_position</code>). For each such spatial point, there will be an argument <code>{spatial_point_position}_bbox</code> (e.g. <code>pt_position_bbox</code>) that accepts a 2x3 list of coordinates specifying the upper and lower bounds for the query. For example, to find all nucleus centroids within a particular bounding box, it would be:</p> <pre><code>bounding_box = [[min_x, min_y, min_z], [max_x, max_y, max_z]]\nnuc_df = client.materialize.tables.nucleus_detection_v0(\n    pt_position_bbox=bounding_box\n).query()\n</code></pre> <p>If you are not using any filters, you can omit the parenthesis and use the <code>get_all</code> or <code>get_all_live</code> functions directly, which act similarly to the <code>query</code> and <code>live_query</code> functions respectively. The first example could be rewritten as:</p> <pre><code>nuc_df = client.materialize.tables.nucleus_detection_v0.get_all()\n</code></pre> <p>If you want to list all available fields, you can use the <code>.fields</code> attribute. Similarly, you can get all numeric fields with the <code>.numeric_fields</code> attribute and all spatial fields (allowing bounding box queries) with <code>.spatial_fields</code>.</p> <pre><code>nuc_df = client.materialize.tables.nucleus_detection_v0.spatial_fields\n</code></pre> <p>If you need to specify the table programmatically, you can also use a dictionary-style approach to getting the table filtering function. For example, an equivalent version of the above line would be:</p> <pre><code>my_ids = [373879, 111162]\nmy_table = 'nucleus_detection_v0'\nnuc_df = client.materialize.tables[my_table](id=my_ids).query()\n</code></pre> <p>The <code>query</code> function can also take arguments relating to timestamps or formatting where they act just like in the other query method. In particular, the arguments that apply to <code>query</code> are: <code>select_columns</code>, <code>offset</code>, <code>limit</code>, <code>split_positions</code>, <code>materialization_version</code>, <code>timestamp</code>, <code>metadata</code>, <code>desired_resolution</code>, and <code>get_counts</code>. For example, to add a desired resolution and split positions in the above query, it would look like:</p> <pre><code>my_ids = [373879, 111162]\nnuc_df = client.materialize.tables.nucleus_detection_v0(\n    id=my_ids\n).query(\n    split_positions=True,\n    desired_resolution=[1,1,1],\n)\n</code></pre> <p>Inequalities can also be used in filtering numeric columns. Here, you can pass a dictionary instead of a list of values, with the keys being inequality operators (\"&gt;\", \"&gt;=\", \"&lt;\", and \"&lt;=\") and the values being the comparison. For example, to query for all nuclei with a volume greater than 1000:</p> <pre><code>client.materialize.tables.nucleus_detection_v0(\n    volume={\"&gt;\": 1000}\n).query()\n</code></pre> <p>You can also use multiple inequalities in the same dictionary to filter within a range. For example, to query for all nuclei with a volume between 500 and 750:</p> <pre><code>client.materialize.tables.nucleus_detection_v0(\n    volume={\"&gt;\": 500, \"&lt;\": 750}\n).query()\n</code></pre> <p>If you want to do a live query instead of a materialized query, the filtering remains identical but we use the <code>live_query</code> function instead. The one required argument for <code>live_query</code> is the timestamp.</p> <pre><code>my_ids = [373879, 111162]\nnuc_df = client.materialize.tables.nucleus_detection_v0(\n    id=my_ids\n).live_query(\n    timestamp=datetime.datetime.now(datetime.timezone.utc),\n)\n</code></pre> <p>The live query functions have similar but slightly different arguments: <code>timestamp</code> (required), <code>offset</code>, <code>limit</code>, <code>split_positions</code>, <code>metadata</code>, <code>desired_resolution</code>, and <code>allow_missing_lookups</code>.</p> <p>Note that way that IPython handles docstrings means that while you can use <code>?</code> to get the docstring of the filtering part of the function, you can't simply do something like <code>client.materialize.tables.nucleus_detection_v0().query?</code>. It will tell you the function can't be found, because technically the <code>query</code> function does not yet exist until the table filtering function is called.</p> <p>Instead, if you want to glimpse the docstring of the query or live_query functions, you need to split it into two lines:</p> <pre><code>qry_func = client.materialize.tables.nucleus_detection_v0().query\nqry_func?\n</code></pre> <p>Finally, if the project you are working with has views, a similar interface is available to them via <code>client.materialize.views</code>. Currently views are not compatible with live query, and so only the <code>.query</code> function is available.</p>"},{"location":"tutorials/schemas/","title":"EM Annotation Schemas","text":"<p>The EMAnnotationSchemas client lets one look up the available schemas and how they are defined. This is mostly used for programmatic interactions between services, but can be useful when looking up schema definitions for new tables.</p>"},{"location":"tutorials/schemas/#get-the-list-of-schema","title":"Get the list of schema","text":"<p>One can get the list of all available schema with the <code>schema</code> method. Currently, new schema have to be generated on the server side, although we aim to have a generic set available to use.</p> <pre><code>client.schema.get_schemas()\n</code></pre>"},{"location":"tutorials/schemas/#view-a-specific-schema","title":"View a specific schema","text":"<p>The details of each schema can be viewed with the <code>schema_definition</code> method, formatted as per JSONSchema.</p> <pre><code>example_schema = client.schema.schema_definition('microns_func_coreg')\nexample_schema\n</code></pre> <p>This is mostly useful for programmatic interaction between services at the moment, but can also be used to inspect the expected form of an annotation by digging into the format.</p> <pre><code>example_schema['definitions']['FunctionalCoregistration']\n</code></pre>"},{"location":"tutorials/skeletonization/","title":"Skeletonization","text":"<p>The skeleton service enables you to generate and retrieve skeletons from the server. This saves you the trouble of performing skeletonization routines on your local machine, while also saving you the trouble of understanding many of the details inside of skeletonization. Furthermore, the server offers the ability to generate multiple skeletons simultaneously, in parallel. And lastly, the server caches each skeleton that is generated so that it only ever needs to be generated once. All subsequent requests for the same skeleton will simply return the cached skeleton with notable turn-around-time improvements.</p>"},{"location":"tutorials/skeletonization/#initializing-the-client","title":"Initializing the client","text":"<p>The simplest way to initialize the CAVEclient is by merely providing the datastack of interest:</p> <pre><code>import caveclient as cc\n\nclient = cc.CAVEclient(&lt;datastack_name&gt;)\n</code></pre> <p>With a CAVEclient built, you can now investigate the current build version of the SkeletonClient:</p> <pre><code>client.skeleton.get_version()\n</code></pre> <p>And you can see a list of available skeleton versions. In most cases you will want to use the highest (most recent) version provided.</p> <pre><code>client.skeleton.get_versions()\n</code></pre>"},{"location":"tutorials/skeletonization/#retrieving-a-skeleton","title":"Retrieving a skeleton","text":"<p>Retrieve a skeleton using <code>get_skeleton()</code>. The simplest usage is:</p> <pre><code>sk = client.skeleton.get_skeleton(\n    &lt;root_id&gt;,\n    output_format=&lt;output_format&gt;,\n)\n</code></pre> <p>where the availale output_formats (described below) are:</p> <ul> <li><code>'dict'</code> (default if unspecified)</li> <li><code>'swc'</code> (a Pandas Dataframe)</li> </ul> <p>If the skeleton doesn't exist in the server cache, it may take 20-60 seconds to generate the skeleton before it is returned. This function will block during that time. Any subsequent retrieval of the same skeleton should go very quickly however.</p> <p>To specify a nondefault skeleton version:</p> <pre><code>sk = client.skeleton.get_skeleton(\n    &lt;root_id&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    output_format=&lt;output_format&gt;,\n)\n</code></pre> <p>To specify a specific datastack:</p> <pre><code>sk = client.skeleton.get_skeleton(\n    &lt;root_id&gt;,\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    output_format=&lt;output_format&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#peering-into-the-contents-of-the-cache","title":"Peering into the contents of the cache","text":"<p>Most end-users shouldn't need to use the following function very much, but to see the contents of the cache for a given root id, set of root ids, root id prefix, or set of prefixes:</p> <pre><code>get_cache_contents(\n    root_id_prefixes=&lt;root_id&gt;,\n)\n</code></pre> <p>You can also add additional parameters as needed:</p> <pre><code>get_cache_contents(\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    root_id_prefixes=&lt;root_id&gt;,\n)\n</code></pre> <p>The primary parameter, <code>root_id_prefixes</code>, can be a list of root ids:</p> <pre><code>get_cache_contents(\n    root_id_prefixes=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>The primary parameter can also be a root id prefix, which will match any associated root ids. Since this could potentially return a large list of results, there is an optional <code>limit</code> parameter so you don't overwhelm the memory of your processing environment, e.g., a Jupyter notebook or some other Python script running on your local machine:</p> <pre><code>get_cache_contents(\n    root_id_prefixes=&lt;root_id_prefix&gt;,\n    limit=&lt;limit&gt;,\n)\n</code></pre> <p>Note that the limit only constraints the size of the return value. The internal operation of the function will still receive the full list when it passes the prefix on to CloudFiles. Consequently, calling this function for a short prefix may block for a long time.</p> <p>And of course you can also pass in a list of prefixes (or a mixture of full ids and partial prefixes):</p> <pre><code>get_cache_contents(\n    root_id_prefixes=[&lt;root_id_prefix&gt;, &lt;root_id_prefix&gt;, ...],\n    limit=&lt;limit&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#querying-the-presence-of-a-skeleton-in-the-cache","title":"Querying the presence of a skeleton in the cache","text":"<p>The function shown above isn't necessarily the most direct way to simply inquire whether a skeleton exists in the cache for a given root id. For that purpose, the following function is better suited:</p> <pre><code>skeletons_exist(\n    root_ids=&lt;root_id&gt;,\n)\n</code></pre> <p>Or:</p> <pre><code>skeletons_exist(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>Note that this function doesn't accept prefixes, as supported by <code>cache_query_contents()</code>. Only full root ides are supported. When querying with as single root id, the return value will be a boolean. When querying with a list of ids, the return value will be a Python dictionary mapping from each id to a boolean.</p> <p>This function also takes the same optional parameters described above:</p> <pre><code>skeletons_exist(\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n    root_ids=&lt;root_id&gt;,  # Or [&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#retrieving-multiple-skeletons","title":"Retrieving multiple skeletons","text":"<p>You can retrieve a large set of skeletons in a single function call:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>If any skeletons are not generated yet, the default behavior is to skip those root ids and only return skeletons that are already available. But you can override this default behavior:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    generate_missing_skeletons=[False|True],\n)\n</code></pre> <p>Any root ids for which skeletonization is required will be generated one at a time, at a cost of 20-60 seconds each. Consequently, there is a hard-coded limit of 10, after which all subsequent missing skeletons will not be returned.</p> <p>By default, skeletons are returned in JSON format. However SWC is also supported, thusly:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    output_format=&lt;\"json\"|\"swc\"&gt;\n)\n</code></pre> <p>And the usual defaults can be overridden again:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#generating-multiple-skeletons-in-parallel","title":"Generating multiple skeletons in parallel","text":"<p><code>get_bulk_skeletons()</code> is not an effective way to produce a large number of skeletons since it operates synchronously, generating one skeleton at a time. In order to generate a large number of skeletons it is better to do so in parallel. The following function dispatches numerous root ids for skeletonization without returning anything immediately. The root ids are then distributed on the server for parallel skeletonization and eventual caching. Once they are in the cache, you can retrieve them. Of course, it can be tricky to know when they are available. That is addressed further below. Here's how to dispatch asynchronous bulk skeletonization:</p> <pre><code>generate_bulk_skeletons_async(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n)\n</code></pre> <p>And with the usual overrides:</p> <pre><code>get_bulk_skeletons(\n    root_ids=[&lt;root_id&gt;, &lt;root_id&gt;, ...],\n    datastack_name=&lt;datastack_name&gt;,\n    skeleton_version=&lt;sk_version&gt;,\n)\n</code></pre>"},{"location":"tutorials/skeletonization/#retrieving-asynchronously-generated-skeletons","title":"Retrieving asynchronously generated skeletons","text":"<p>In order to retrieve asynchronously generated skeletons, it is necessary to poll the cache for the availability of the skeletons and then eventually retrieve them. Here's an example of such a workflow:</p> <pre><code># Dispatch multiple asynchronous, parallel skeletonization and caching processes\ngenerate_bulk_skeletons_async(root_ids)\n\n# Repeatedly query the cache for the existence of the skeletons until they are all available\nwhile True:\n    skeletons_that_exist = client.skeleton.skeletons_exist(root_ids=rids)\n    num_skeletons_found = sum(skeletons_that_exist.values())\n    if num_skeletons_found == len(rids):\n        break\n    sleep(10)  # Pause for ten seconds and check again\n\n# Retrieve the skeletons (remember, SWC is also offered)\nskeletons_json = get_bulk_skeletons(root_ids)\n</code></pre>"},{"location":"tutorials/state/","title":"JSON Neuroglancer State Service","text":"<p>We store the JSON description of a Neuroglancer state in a simple database at the JSON Service. This is a convenient way to build states to distribute to people, or pull states to parse work by individuals. The JSON Client is at <code>client.state</code></p> <pre><code>client.state\n</code></pre>"},{"location":"tutorials/state/#retrieving-a-state","title":"Retrieving a state","text":"<p>JSON states are found simply by their ID, which you get when uploading a state. You can download a state with <code>get_state_json</code>.</p> <pre><code>example_id = 4845531975188480\nexample_state = client.state.get_state_json(test_id)\nexample_state['layers'][0]\n</code></pre>"},{"location":"tutorials/state/#uploading-a-state","title":"Uploading a state","text":"<p>You can also upload states with <code>upload_state_json</code>. If you do this, the state id is returned by the function. Note that there is no easy way to query what you uploaded later, so be VERY CAREFUL with this state id if you wish to see it again.</p> <p>Note: If you are working with a Neuroglancer Viewer object or similar, in order to upload, use viewer.state.to_json() to generate this representation.</p> <pre><code>example_state['layers'][0]['name'] = 'example_name'\nnew_id = client.state.upload_state_json(example_state)\n</code></pre> <pre><code>test_state = client.state.get_state_json(new_id)\ntest_state['layers'][0]['name']\n</code></pre>"},{"location":"tutorials/state/#generating-a-neuroglancer-url","title":"Generating a Neuroglancer URL","text":"<p>Once you have a state ID, you want to turn it into a well-formatted link. So you don\\'t have to remember all the endpoints, we can do this from the state client.</p> <pre><code>ngl_base = 'neuromancer-seung-import.appspot.com'\nclient.state.build_neuroglancer_url(new_id, ngl_base)\n</code></pre> <p>Note that the neuroglancer base can be found in the info service under <code>client.info.viewer_site()</code>.</p>"}]}
\ No newline at end of file