Skip to content

Commit

Permalink
Adds a little to docs on state persistence
Browse files Browse the repository at this point in the history
This mostly adds TODOs but it sets up the structure.
  • Loading branch information
skrawcz committed Mar 16, 2024
1 parent dc24d72 commit c24440e
Show file tree
Hide file tree
Showing 10 changed files with 64 additions and 8 deletions.
24 changes: 24 additions & 0 deletions docs/concepts/state-persistence.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
=================
State Persistence
=================

.. _state-persistence:

Key to writing a real life `burr` application is state persistence. E.g. you're building a chat bot and you
want to store the conversation history and then reload it when you restart. Or, you have a long running process,
or series of agents, and you want to store the state of the process after each action, and then reload it if it fails, etc.

Here we'll walk through how to add state persistence to a `burr` application. The following walk through the relevant
ApplicationBuilder() methods, and then a full toy example.

.initialize_from() method
_________________________
TODO:

.with_persister() method
________________________
TODO:

Supported Persisters
--------------------
TODO:
6 changes: 3 additions & 3 deletions docs/concepts/state.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,6 @@ If you're used to thinking about version control, this is a bit like a commit/ch

Reloading Prior State
---------------------
Note, if state is serializeable, it means that if stored, it can be reloaded. This is useful for
reloading state from a previous run (for debugging or as part of the application), or for storing state in a database.
We are building more capabilties here, for now for debugging purposes, see the :ref:`tracking <trackingclientref>` section.
Note, if state is serializable, it means that if stored, it can be reloaded. This is useful for
reloading state from a previous invocation (for debugging or as part of the application), or for storing state in a database.
We have capabilities here, see the :ref:`state-persistence <state-persistence>` section.
5 changes: 5 additions & 0 deletions docs/examples/ml_training.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,9 @@
ML Model Training
====================

Hyperparameter Optimization is a key part of training machine learning models. For more sophisticated examples that
aren't grid search, `burr` is a quick and easy way to implement hyperparameter optimization. You want to keep
track of hyperparameters and metrics, feeding these into a decision function to determine where to look next,
and `burr` can help you write and manage that process.

This is still a WIP -- come back soon, or `contribute your own example <https://github.com/DAGWorks-Inc/burr/issues/new>`_!
3 changes: 3 additions & 0 deletions docs/examples/simulation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,7 @@
Simulation
==========

Simulations require state to be maintained across time steps. Burr provides a simple way for you to construct,
manage, and introspect the state of your simulation.

This is still a WIP -- come back soon, or `contribute your own example <https://github.com/DAGWorks-Inc/burr/issues/new>`_!
2 changes: 1 addition & 1 deletion docs/getting_started/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,6 @@ along with a fully built server.

.. code-block:: bash
pip install burr[learn]
pip install burr[start]
This will give you tools to visualize, track, and interact with the UI. Note, if you're using ``zsh``, you'll need to add quotes around the install target, (``pip install "burr[learn]"``).
4 changes: 2 additions & 2 deletions docs/getting_started/simple-example.rst
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ Next, let's put together our application. To do this, we'll use an ``Application
("count", "count", expr("counter < 10")), # Keep counting if the counter is less than 10
("count", "done", default) # Otherwise, we're done
).with_entrypoint("count") # we have to start somewhere
.with_tracker("my_first_app")
.with_tracker(project="my_first_app")
.build()
)
Expand Down Expand Up @@ -124,7 +124,7 @@ If you want to copy/paste, you can open up the following code block and add to a
("count", "count", expr("counter < 10")), # Keep counting if the counter is less than 10
("count", "done", default) # Otherwise, we're done
).with_entrypoint("count") # we have to start somewhere
.with_tracker("my_first_app")
.with_tracker(project="my_first_app")
.build()
)
app.visualize("./graph", format="png", include_conditions=True, include_state=True)
Expand Down
2 changes: 2 additions & 0 deletions docs/main.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ Burr

Welcome to Burr's documentation, the state machine for managing complex data/LLM projects.

TODO: VIDEO.

You'll find this documentation separated into three sections.

- If you don't know where to start, go to :ref:`getting started <gettingstarted>`.
Expand Down
1 change: 1 addition & 0 deletions docs/reference/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ need functionality that is not publicly exposed, please open an issue and we can
application
actions
state
persister
conditions
tracking
visibility
Expand Down
21 changes: 21 additions & 0 deletions docs/reference/persister.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
=================
State Persistence
=================

This is an interface one should implement to provide state persistence for your
application.

.. autoclass:: burr.core.state.BasicStatePersistence
:members:

.. automethod:: __init__


Supported Implementations
=========================


.. autoclass:: burr.core.state.SQLLitePersistence
:members:

.. automethod:: __init__
4 changes: 2 additions & 2 deletions docs/reference/tracking.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
Tracking
========

Reference on the Tracking API. This is purely for information -- you should not use this directly.
Rather, you should use this through :py:meth:`burr.core.application.ApplicationBuilder.with_tracker`
Reference on the Tracking/Telemetry API.
Rather, you should use this throug/in conjunction with :py:meth:`burr.core.application.ApplicationBuilder.with_tracker`.


.. autoclass:: burr.tracking.LocalTrackingClient
Expand Down

0 comments on commit c24440e

Please sign in to comment.