v1.8.0

Substreams

Capacity Management

• Integrated the GlobalRequestPool service in the Tier1App to manage global requests pooling.

• Integrated the GlobalWorkerPool service in the Tier1App to manage global worker pooling.

• Added flag substreams-tier1-global-worker-pool-address, the address of the global worker pool to use for the substreams tier1. (disabled if empty)

• Added flag substreams-tier1-global-worker-pool-keep-alive-delay delay between two keep alive call to the global worker pool. Default is 25s")

• Added flag substreams-tier1-global-request-pool-keep-alive-delay delay between two keep alive call to the global worker pool for request. Default is 25s

• Added flag substreams-tier1-default-max-request-per-user default max request per user, this will be use of the global worker pool is not reachable. Default is 5

• Added flag substreams-tier1-default-minimal-request-life-time-second default minimal request life time, this will be use of the global worker pool is not reachable. . Default is 180

• Limit parallel execution of a stage's layer: Previously, the engine was executing modules in a stage's layer all in parallel.
  We now change that behavior, development mode will from now on execute every sequentially and when in production mode will
  limit parallelism to 2 (hard-coded) for now.
  The auth plugin can control that value dynamically by providing a trusted header X-Sf-Substreams-Stage-Layer-Parallel-Executor-Max-Count.

Performance

• Fixed a regression since "v1.7.3" where the SkipEmptyOutput instruction was ignored in substreams mappers
• Add shared cache for tier1 execution near HEAD, to prevent multiple tier1 instances from reprocessing the same module on the same block when it comes in (ex: foundational modules)
• Improved fetching of state caches on tier1 requests to speed up "time to first data"

Tools

• make 'compare-blocks' command support one-blocks stores as well as merged-blocks