feat(cli)!: find bots by default paths

[johnson2427]

What I did

In an attempt to make the dev ux simpler, I allow for the path to be an option. If the path does not exist, we set the bath to pathlib.Path.cwd()/bots/bot.py

We check and raise if the file and/or the directory does not exist.

We are also adding a Dockerfile generator to the _cli.py file that will search the bots/ directory for filenames. Each filename will be generated as a Dockerfile.<filename>. The Dockerfile will copy the filename into the bots/ directory as bot.py.

fixes: #122
fixes: #123
fixes: SBK-564
fixes: SBK-563

How I did it

Made minor changes to the run command and adding a method to generate the dockerfiles.

How to verify it

Without a bots/ directory in the root of the project run:

silverback run --network ethereum:mainnet

This will raise and show a:

FileNotFoundError: The bots directory '/path/to/silverback/bots' does not exist. You should have a `bots/` folder in the root of your project.

Then:

mkdir bots
silverback run --network ethereum:mainnet

This will raise and show:

FileNotFoundError: The bot.py file does not exist in the bots directory.

You can then run:

touch bots/bot.py
silverback run --network ethereum:mainnet

And you will get an error that does not relate to files missing.
You can also run:

touch bots/some_file.py
silverback run bots.some_file:bot

And you will get an error that does not pertain to the files/directory missing.

MORE TO BE ADDED

Checklist

• Passes all linting checks (pre-commit and CI jobs)
• New test cases have been added and are passing
• Documentation has been updated
• PR title follows Conventional Commit standard (will be automatically included in the changelog)

[linear]

SBK-564 feat: auto-build the `silverback run` path for simpler dev ux

SBK-563 feat: add dockerfile generation method

[johnson2427]

@fubuloubu We should probably inform the developers that each file in the bots/ folder should be independent of one another. And that each file should be its own bot. I can add a long note in the README about this

[fubuloubu]

@fubuloubu We should probably inform the developers that each file in the bots/ folder should be independent of one another. And that each file should be its own bot. I can add a long note in the README about this

You can probably raise a warning if you see __init__.py and that will prevent any cross-coupling

But docs definitely encouraged for the best way to package a module relative to the bots folder (as non-local import)

[fubuloubu]

NOTE: marking breaking just due to the docs updates needed, and the changing expectations on how to write an app (which should get more widely noted w/ Silverback v0.6.0 alongside full cluster beta access work)

[fubuloubu]

One suggestion, looks good to me!

Needs some local testing from others (unsure if anyone else has checked it out yet)

[Ninjagod1251]

I was able to run it with a simple update shareprice bot.py via silverback run --network ethereum:mainnet:alchemy and .env file with alchemy, vault address, and etherescan :

import os

from ape import Contract, chain, networks
from silverback import SilverbackApp


bot = SilverbackApp()

with networks.ethereum.mainnet.use_provider("alchemy"):  # or "infura"
    vault = Contract(os.environ.get("ERC4626_VAULT_ADDRESS"))

one_share = 10 ** vault.decimals()


@bot.on_(chain.blocks)
def update_shareprice(_):
    """
    Add price to database (Update database)
    """
    price = vault.convertToShares(one_share) / one_share
    print(f"Price Event: {price}")

result:

(ape) (botenv) chris@DESKTOP-ID4V0R6 ~/_src/silverback (feat/generate-images)$ silverback run --network ethereum:mainnet:alchemy
INFO: Loading Silverback App with settings:
  APP_NAME="bot"
  BROKER_CLASS="taskiq:InMemoryBroker"
  NETWORK_CHOICE="ethereum:mainnet:alchemy"
SUCCESS: Loaded Silverback App:
  NETWORK="ethereum:mainnet"
  SIGNER=None
INFO: Using WebsocketRunner: max_exceptions=3
INFO: Worker using Silverback SDK v0.5.9, available task types:
- user:shutdown
- user:new-block
- system:user-taskdata
- system:user-all-taskdata
- system:config
- user:event-log
- user:startup
INFO: update_shareprice[block_number=20887599,block_hash=0x3e2a203719d6a277c5105ad831b7b72bf8179b5604c51e322503080fd73fdf21] - Started
Price Event: 0.9550524926100307
SUCCESS: update_shareprice[block_number=20887599,block_hash=0x3e2a203719d6a277c5105ad831b7b72bf8179b5604c51e322503080fd73fdf21] - 0.310s (2.4%)
INFO: update_shareprice[block_number=20887600,block_hash=0xe6edc0afffb6f65df6372d0d5c3355b2a4554f96155b06d750c991f29dd666ed] - Started
Price Event: 0.9550524806676409
SUCCESS: update_shareprice[block_number=20887600,block_hash=0xe6edc0afffb6f65df6372d0d5c3355b2a4554f96155b06d750c991f29dd666ed] - 0.260s (2.0%)
INFO: update_shareprice[block_number=20887601,block_hash=0x71614a308078a927806dc4266f25857cf93ce7afe061869856d52ae8cc721327] - Started
Price Event: 0.9550524687252515
SUCCESS: update_shareprice[block_number=20887601,block_hash=0x71614a308078a927806dc4266f25857cf93ce7afe061869856d52ae8cc721327] - 0.320s (2.5%)
INFO: update_shareprice[block_number=20887602,block_hash=0xa44c2e9e27674a0550911a30c878b5d0c9480e013abeb4df45caa77a3f36cf0a] - Started
Price Event: 0.9550524567828624
SUCCESS: update_shareprice[block_number=20887602,block_hash=0xa44c2e9e27674a0550911a30c878b5d0c9480e013abeb4df45caa77a3f36cf0a] - 0.280s (2.2%)
INFO: update_shareprice[block_number=20887603,block_hash=0x5339db03fcb697cff2e3779430760eae35cd9a70b7f083bce87ac92a9f5a0ee3] - Started

[fubuloubu]

Overall, I think that the silverback build command should be producing an image if it can, by wrapping docker (if available) in a particular "expected" way to build/tag the images e.g.

$ silverback build --generate
Generating './images/Dockerfile.botA'
Generating './images/Dockerfile.botB'
Generating './images/Dockerfile.botC'
Building './images/Dockerfile.botA' as 'github-org/repo-name-botA:latest'
ERROR: Command 'docker' not found, cannot build image

This would build with the tag github-org/repo-botA:latest (if git is available, and you can find a github remote)

...but this might be getting into stuff that is best left to the github action to configure (basically make the GH action call this command and just keep this command simple with some sensible defaults)

[johnson2427]

I can do this, I really don't want to add another dependency to silverback though, so I'd use system level code to interact with the docker.sock. We don't normally do that though, so up to you. Like you said, might be getting into stuff that would be best to leave to the user

[fubuloubu]

sure, I think in that case, silverback build might not be the right name for the command (originally I was thinking silverback build --generate or some variant to show that it was creating docker files first)

I wouldn't really call it a dependency though, it's either docker command exists and we subprocess out to it in a specific way, or it doesn't and you raise an error and exit

[fubuloubu]

Remember, we are just handling suggested behaviors here to make it easy for folks. Anything more complicated should be done manually, the silverback subcommand only works if you follow those suggestions

[johnson2427]

I like the silverback buld terminology. I can add the option for --generate if you'd like. I really think we should get this out, then I can make a PR following this one to really dive into the build functionality to add the remainder of what we want here. Current functionality allows me to do everything we want to in the workflow

[fubuloubu]

I like the silverback buld terminology. I can add the option for --generate if you'd like. I really think we should get this out, then I can make a PR following this one to really dive into the build functionality to add the remainder of what we want here. Current functionality allows me to do everything we want to in the workflow

The ideal workflow here is the average person doesn't have to touch docker command at all if they follow our suggestions and have docker installed

This will be what our github action will do e.g. run: silverback build --generate (which makes it easy to debug the github action if it is not working, just run same command locally)

[johnson2427]

just to be clear, we want silverback build to run docker build for each file in the .silverback-images directory. And then we want silverback build --generate to create the dockerfiles in the .silverback-images directory. Correct?

[fubuloubu]

just to be clear, we want silverback build to run docker build for each file in the .silverback-images directory. And then we want silverback build --generate to create the dockerfiles in the .silverback-images directory. Correct?

Yes!

[mikeshultz]

Maybe we can separate these out? It seems very likely that all users will need to manually fuck with their dockerfiles. Adding dependencies or data files. So maybe this should be two steps.

1. Generate dockerfile templates for the known bot sources
2. Build images for said bots

Or maybe add a --build option that specifically runs docker build.

[johnson2427]

@fubuloubu let me know if you want to add a build flag. I think the current workflow is good, but I'd like your input here

[mikeshultz]

Ship it!