Skip to content

Latest commit

 

History

History
46 lines (29 loc) · 1.51 KB

README.md

File metadata and controls

46 lines (29 loc) · 1.51 KB

Dev setup

Note: Run these commands from the root of the project

Run redis, postgres and prometheus docker containers

docker compose up db redis prometheus

Seed the database and run migrations

INIT_FILE_PATH=./init.sql sea migrate up --database-url=postgres://solana:solana@localhost:5432/solana

Config for grpc2redis ./config-grpc2redis.yml

Run grpc2redis service

This service will listen to geyser gRPC account and transaction updates. It makes multiple subscriptions to the gRPC stream and filter the data based on the config. The data (vec of bytes) is pushed to a pipeline and then flushed to redis at regular intervals.

Note: Log level can be set to info, debug, warn, error

RUST_LOG=info cargo run --bin das-grpc-ingest  -- --config grpc-ingest/config-grpc2redis.yml grpc2redis

Config for Ingester ./config-ingester.yml

Run the Ingester service

This service performs many concurrent tasks

  • Fetch account updates from redis and process them using using program_transformer
  • Fetch transaction updates from redis and processe them
  • Fetch snapshots from redis and process them
  • download token metedata json and store them in postgres db
 RUST_LOG=debug,sqlx=warn cargo run --bin das-grpc-ingest  -- --config grpc-ingest/config-ingester.yml ingester

Metrics

Both grpc2redis and ingester services expose prometheus metrics and can be accessed at http://localhost:9090/metrics