feat!: add app.state (#80)

* refactor!: change AppState to StateSnapshot to better express it

* feat!: add `app.state`

Adds 2 new system tasks to load and store a snapshot of state.
Currently this is just `last_block_processed` and `last_block_seen`.

This paves the way for the `Parameter` feature, but does not include
that just yet.

BREAKING CHANGE: state snapshotting migrates from runner to worker

* refactor: update from peer review

* docs: add more color in docs for `app.state`

Also updated docstrings within `silverback/application.py`

* fix: don't raise error on exit, only note the error

* refactor: have runner load and save the snapshot in datastore

* docs: add a note to example about using worker startup event

* docs: show off `app.state` and `CircuitBreaker` with more common example

* docs: add another commit about triggering a txn to userguide

* refactor(middlware): don't display INFO logs for system task start

* feat!: change SilverbackApp to SilverbackBot and all changes surrounding it

* fix: dockerfile

* fix: bad isort

* fix: docs surrounding app vs bot

* fix: minor rewording in development docs

* fix: add taskiq-redis

* fix: typo

---------

Co-authored-by: johnson2427 <[email protected]>

Dockerfile

@@ -16,11 +16,14 @@ RUN pip wheel . --wheel-dir=/wheels
 # Install from wheels
 FROM ghcr.io/apeworx/ape:${BASE_APE_IMAGE_TAG:-latest}
 USER root
-COPY --from=builder /wheels /wheels
+COPY --from=builder /wheels/*.whl /wheels
 RUN pip install --upgrade pip \
-    && pip install silverback \
+    && pip install \
+    --no-cache-dir --find-links=/wheels \
     'taskiq-sqs>=0.0.11' \
-    --no-cache-dir --find-links=/wheels
+    'taskiq-redis>=1.0.2,<2' \
+    silverback
+
 USER harambe
 
 ENTRYPOINT ["silverback"]

README.md

@@ -1,19 +1,19 @@
 # Quick Start
 
 Silverback lets you create and deploy your own Python bots that respond to on-chain events.
-The Silverback library leverages the [Ape](https://docs.apeworx.io/ape/stable/userguides/quickstart) development framework as well as it's ecosystem of plugins and packages to enable you to develop simple-yet-sophisticated automated applications that can listen and respond to live chain data.
+The Silverback library leverages the [Ape](https://docs.apeworx.io/ape/stable/userguides/quickstart) development framework as well as it's ecosystem of plugins and packages to enable you to develop simple-yet-sophisticated automated bots that can listen and respond to live chain data.
 
-Silverback applications are excellent for use cases that involve continuously monitoring and responding to on-chain events, such as newly confirmed blocks or contract event logs.
+Silverback bots are excellent for use cases that involve continuously monitoring and responding to on-chain events, such as newly confirmed blocks or contract event logs.
 
-Some examples of these types of applications:
+Some examples of these types of bots:
 
 - Monitoring new pool creations, and depositing liquidity
 - Measuring trading activity of popular pools
 - Listening for large swaps to update a telegram group
 
 ## Documentation
 
-Please read the [development userguide](https://docs.apeworx.io/silverback/stable/userguides/development.html) for more information on how to develop an application.
+Please read the [development userguide](https://docs.apeworx.io/silverback/stable/userguides/development.html) for more information on how to develop a bot.
 
 ## Dependencies
 
@@ -72,11 +72,11 @@ Silverback will automatically register files in this folder as separate bots tha
 
 ```{note}
 It is also suggested that you treat this as a scripts folder, and do not include an __init__.py
-If you have a complicated project, follow the previous example to ensure you run the application correctly.
+If you have a complicated project, follow the previous example to ensure you run the bot correctly.
 ```
 
 ```{note}
-A final suggestion would be to name your `SilverbackApp` object `bot`. Silverback automatically searches 
+A final suggestion would be to name your `SilverbackBot` object `bot`. Silverback automatically searches 
 for this object name when running. If you do not do so, once again, ensure you replace `example` with 
 `example:<name-of-object>` the previous example.
 ```
@@ -139,7 +139,7 @@ Traceback (most recent call last):
 ape_alchemy.exceptions.MissingProjectKeyError: Must set one of $WEB3_ALCHEMY_PROJECT_ID, $WEB3_ALCHEMY_API_KEY, $WEB3_ETHEREUM_MAINNET_ALCHEMY_PROJECT_ID, $WEB3_ETHEREUM_MAINNET_ALCHEMY_API_KEY.
 ```
 
-Go to [Alchemy](https://alchemy.com), create an account, then create an application in their dashboard, and copy the API Key.
+Go to [Alchemy](https://alchemy.com), create an account, then create an bot in their dashboard, and copy the API Key.
 
 Another requirement for the command from `Docker Usage` to run the given example is that it uses [ape-tokens](https://github.com/ApeWorX/ape-tokens) plugin to look up token interfaces by symbol.
 In order for this to work, you should have installed and configured that plugin using a token list that includes both YFI and USDC on Ethereum mainnet.

docs/userguides/development.md

@@ -1,6 +1,6 @@
-# Developing Applications
+# Developing Bots
 
-In this guide, we are going to show you more details on how to build an application with Silverback.
+In this guide, we are going to show you more details on how to build an bot with Silverback.
 
 ## Prerequisites
 
@@ -15,15 +15,15 @@ There are 3 suggested ways to structure your project. In the root directory of y
 
 2. Create a `bots/` folder. Then develop bots in this folder as separate scripts (Do not include a __init__.py file).
 
-3. Create a `bot/` folder with a `__init__.py` file that will include the instantiation of your `SilverbackApp()` object.
+3. Create a `bot/` folder with a `__init__.py` file that will include the instantiation of your `SilverbackBot()` object.
 
 The `silverback` cli automatically searches for python scripts to run as bots in specific locations relative to the root of your project.
 It will also be able to detect the scripts inside your `bots/` directory and let you run those by name (in case you have multiple bots in your project).
 
 If `silverback` finds a module named `bot` in the root directory of the project, then it will use that by default.
 
 ```{note}
-It is suggested that you create the instance of your `SilverbackApp()` object by naming the variable `bot`, since `silverback` will autodetect that variable name when loading your script file.
+It is suggested that you create the instance of your `SilverbackBot()` object by naming the variable `bot`, since `silverback` will autodetect that variable name when loading your script file.
 ```
 
 Another way you can structure your bot is to create a `bot` folder and define a runner inside of that folder as `__init__.py`.
@@ -43,7 +43,7 @@ If your bot's module name is `example.py` (for example), you can run it like thi
 silverback run example --network your:network:of:choice
 ```
 
-If the variable that you call the `SilverbackApp()` object is something other than `bot`, you can specific that by adding `:{variable-name}`:
+If the variable that you call the `SilverbackBot()` object is something other than `bot`, you can specific that by adding `:{variable-name}`:
 
 ```bash
 silverback run example:my_bot --network your:network:of:choice
@@ -52,7 +52,7 @@ silverback run example:my_bot --network your:network:of:choice
 We will automatically detect all scripts under the `bots/` folder automatically, but if your bot resides in a location other than `bots/` then you can use this to run it:
 
 ```bash
-silverback run folder.example:app --network your:network:of:choice
+silverback run folder.example:bot --network your:network:of:choice
 ```
 
 Note that with a `bot/__init__.py` setup, silverback will also autodetect it, and you can run it with:
@@ -69,21 +69,21 @@ For the most streamlined experience, develop your bots as scripts, and avoid rel
 If you follow these suggestions, your Silverback deployments will be easy to use and require almost no thought.
 ```
 
-## Creating an Application
+## Creating a Bot
 
-Creating a Silverback Application is easy, to do so initialize the `silverback.SilverbackApp` class:
+Creating a Silverback Bot is easy, to do so initialize the `silverback.SilverbackBot` class:
 
 ```py
-from silverback import SilverbackApp
+from silverback import SilverbackBot
 
-bot = SilverbackApp()
+bot = SilverbackBot()
 ```
 
-The SilverbackApp class handles state and configuration.
+The SilverbackBot class handles state and configuration.
 Through this class, we can hook up event handlers to be executed each time we encounter a new block or each time a specific event is emitted.
-Initializing the app creates a network connection using the Ape configuration of your local project, making it easy to add a Silverback bot to your project in order to perform automation of necessary on-chain interactions required.
+Initializing the bot creates a network connection using the Ape configuration of your local project, making it easy to add a Silverback bot to your project in order to perform automation of necessary on-chain interactions required.
 
-However, by default an app has no configured event handlers, so it won't be very useful.
+However, by default an bot has no configured event handlers, so it won't be very useful.
 This is where adding event handlers is useful via the `bot.on_` method.
 This method lets us specify which event will trigger the execution of our handler as well as which handler to execute.
 
@@ -161,9 +161,9 @@ def block_handler(block, context: Annotated[Context, TaskiqDepends()]):
     ...
 ```
 
-### Application Events
+### Bot Events
 
-You can also add an application startup and shutdown handler that will be **executed once upon every application startup**.  This may be useful for things like processing historical events since the application was shutdown or other one-time actions to perform at startup.
+You can also add an bot startup and shutdown handler that will be **executed once upon every bot startup**.  This may be useful for things like processing historical events since the bot was shutdown or other one-time actions to perform at startup.
 
 ```py
 @bot.on_startup()
@@ -180,7 +180,46 @@ def handle_on_shutdown():
     ...
 ```
 
-*Changed in 0.2.0*: The behavior of the `@bot.on_startup()` decorator and handler signature have changed.  It is now executed only once upon application startup and worker events have moved on `@bot.on_worker_startup()`.
+*Changed in 0.2.0*: The behavior of the `@bot.on_startup()` decorator and handler signature have changed.  It is now executed only once upon bot startup and worker events have moved on `@bot.on_worker_startup()`.
+
+## Bot State
+
+Sometimes it is very useful to have access to values in a shared state across your workers.
+For example you might have a value or complex reference type that you wish to update during one of your tasks, and read during another.
+Silverback provides `bot.state` to help with these use cases.
+
+For example, you might want to pre-populate a large dataframe into state on startup, keeping that dataframe in sync with the chain through event logs,
+and then use that data to determine a signal under which you want trigger transactions to commit back to the chain.
+Such an bot might look like this:
+
+```py
+@bot.on_startup()
+def create_table(startup_state):
+    df = contract.MyEvent.query(..., start_block=startup_state.last_block_processed)
+    ...  # Do some further processing on df
+    bot.state.table = df
+
+
+@bot.on_(contract.MyEvent)
+def update_table(log):
+    bot.state.table = ...  # Update using stuff from `log`
+
+
+@bot.on_(chain.blocks)
+def use_table(blk):
+    if bot.state.table[...].mean() > bot.state.table[...].sum():
+        # Trigger your bot to send a transaction from `bot.signer`
+        contract.myMethod(..., sender=bot.signer)
+    ...
+```
+
+```{warning}
+You can use `bot.state` to store any python variable type, however note that the item is not networked nor threadsafe so it is not recommended to have multiple tasks write to the same value in state at the same time.
+```
+
+```{note}
+Bot startup and bot runtime event triggers (e.g. block or event container) are handled distinctly and can be trusted not to execute at the same time.
+```
 
 ### Signing Transactions
 
@@ -192,10 +231,10 @@ While not recommended, you can use keyfile accounts for automated signing.
 See [this guide](https://docs.apeworx.io/ape/stable/userguides/accounts.html#automation) to learn more about how to do that.
 ```
 
-## Running your Application
+## Running your Bot
 
 Once you have programmed your bot, it's really useful to be able to run it locally and validate that it does what you expect it to do.
-To run your bot locally, we have included a really useful cli command [`run`](../commands/run) that takes care of connecting to the proper network, configuring signers (using your local Ape accounts), and starting up the application client and in-memory task queue workers.
+To run your bot locally, we have included a really useful cli command [`run`](../commands/run) that takes care of connecting to the proper network, configuring signers (using your local Ape accounts), and starting up the bot client and in-memory task queue workers.
 
 ```sh
 # Run your bot on the Ethereum Sepolia testnet, with your own signer:
@@ -206,20 +245,20 @@ $ silverback run my_bot --network :sepolia --account acct-name
 `my_bot:bot` is not required for silverback run if you follow the suggested folder structure at the start of this page, you can just call it via `my_bot`.
 ```
 
-It's important to note that signers are optional, if not configured in the application then `bot.signer` will be `None`.
-You can use this in your application to enable a "test execution" mode, something like this:
+It's important to note that signers are optional, if not configured in the bot then `bot.signer` will be `None`.
+You can use this in your bot to enable a "test execution" mode, something like this:
 
 ```py
 # Compute some metric that might lead to creating a transaction
 if bot.signer:
-    # Execute a transaction via `sender=app.signer`
+    # Execute a transaction via `sender=bot.signer`
 else:
     # Log what the transaction *would* have done, had a signer been enabled
 ```
 
 ```{warning}
-If you configure your application to use a signer, and that signer signs anything given to it, remember that you can lose substational amounts of funds if you deploy this to a production network.
-Always test your applications throughly before deploying, and always use a dedicated key for production signing with your application in a remote setting.
+If you configure your bot to use a signer, and that signer signs anything given to it, remember that you can lose substational amounts of funds if you deploy this to a production network.
+Always test your bots throughly before deploying, and always use a dedicated key for production signing with your bot in a remote setting.
 ```
 
 ```{note}
@@ -230,7 +269,7 @@ Use segregated keys and limit your risk by controlling the amount of funds that
 ### Distributed Execution
 
 Using only the `silverback run ...` command in a default configuration executes everything in one process and the job queue is completely in-memory with a shared state.
-In some high volume environments, you may want to deploy your Silverback application in a distributed configuration using multiple processes to handle the messages at a higher rate.
+In some high volume environments, you may want to deploy your Silverback bot in a distributed configuration using multiple processes to handle the messages at a higher rate.
 
 The primary components are the client and workers.  The client handles Silverback events (blocks and contract event logs) and creates jobs for the workers to process in an asynchronous manner.
 
@@ -265,10 +304,10 @@ silverback worker -w 2
 
 The client will send tasks to the 2 worker subprocesses, and all task queue and results data will be go through Redis.
 
-## Testing your Application
+## Testing your Bot
 
 TODO: Add backtesting mode w/ `silverback test`
 
-## Deploying your Application
+## Deploying your Bot
 
-Check out the [Platform Deployment Userguide](./platform.html) for more information on how to deploy your application to the [Silverback Platform](https://silverback.apeworx.io).
+Check out the [Platform Deployment Userguide](./platform.html) for more information on how to deploy your bot to the [Silverback Platform](https://silverback.apeworx.io).

docs/userguides/platform.md

@@ -1,14 +1,14 @@
-# Deploying Applications
+# Deploying Bots
 
 In this guide, we are going to show you more details on how to deploy your application to the [Silverback Platform](https://silverback.apeworx.io).
 
 ## Creating a Cluster
 
-The Silverback Platform runs your Applications (or "Bots") on dedicated managed application Clusters.
+The Silverback Platform runs your Bots on dedicated managed application Clusters.
 These Clusters will take care to orchestrate infrastructure, monitor, run your triggers, and collect metrics for your applications.
 Each Cluster is bespoke for an individual or organization, and isolates your applications from others on different infrastructure.
 
-Before we deploy our Application, we have to create a Cluster.
+Before we deploy our Bot, we have to create a Cluster.
 If you haven't yet, please sign up for Silverback at [https://silverback.apeworx.io](https://silverback.apeworx.io).
 
 Once you have signed up, you can actually create (and pay for) your Clusters from the Silverback CLI utility by first
@@ -44,7 +44,7 @@ For instance, to list all your available bots on your cluster, use [`silverback
 To obtain general information about your cluster, just use [`silverback cluster info`][silverback-cluster-info],
 or [`silverback cluster health`][silverback-cluster-health] to see the current status of your Cluster.
 
-If you have no bots, we will first have to containerize our Applications and upload them to a container registry that our Cluster is configured to access.
+If you have no bots, we will first have to containerize our Bots and upload them to a container registry that our Cluster is configured to access.
 
 ```{note}
 Building a container for your application can be an advanced topic, we have included the `silverback build` subcommand to help assist in generating Dockerfiles.
@@ -108,7 +108,7 @@ Silverback Clusters include an environment variable management system for exactl
 which you can manage using [`silverback cluster vars`][silverback-cluster-vars] subcommand.
 
 The environment variable management system makes use of a concept called "Variable Groups" which are distinct collections of environment variables meant to be used together.
-These variable groups will help in managing the runtime environment of your Applications by allowing you to segregate different variables depending on each bot's needs.
+These variable groups will help in managing the runtime environment of your Bots by allowing you to segregate different variables depending on each bot's needs.
 
 To create an environment group, use the [`silverback cluster vars new`][silverback-cluster-vars-new] command and give it a name and a set of related variables.
 For instance, it may make sense to make a group of variables for your favorite Ape plugins or services, such as RPC Providers, Blockchain Data Indexers, Etherscan, etc.
@@ -199,7 +199,7 @@ Any task execution that experiences an error will abort execution (and therefore
 All errors encountered during task exeuction are reported to the Cluster for later review by any users with appriopiate access.
 Tasks do not retry (by default), but updates to `app.state` are maintained up until the point an error occurs.
 
-It is important to keep track of these errors and ensure that none of them are in fact critical to the operation of your Application,
+It is important to keep track of these errors and ensure that none of them are in fact critical to the operation of your Bot,
 and to take corrective or preventative action if it is determined that it should be treated as a more critical failure condition.
 ```