Observer API backend

[ch1bo]

Allows for multiple hydra-chain-observer instances to report into one hydra-explorer:

flowchart LR

  subgraph hydra 0.19.0
    plutus-19[hydra-plutus]
    tx-19[hydra-tx]
    node-19[hydra-node]
    observer-19-preview[hydra-chain-observer 0.19.0]
    plutus-19 --> tx-19
    tx-19 --> observer-19-preview
    tx-19 --> node-19
    node-19 --> observer-19-preview
  end
  preview -. chain sync .-> observer-19-preview

  subgraph hydra 0.20.0
    plutus-20[hydra-plutus]
    tx-20[hydra-tx]
    node-20[hydra-node]
    observer-20[hydra-chain-observer 0.20.0]
    plutus-20 --> tx-20
    tx-20 --> observer-20
    tx-20 --> node-20
    node-20 --> observer-20
  end
  mainnet -. chain sync .-> observer-20

  observer-19[hydra-chain-observer 0.19.0]
  mainnet -. chain sync .-> observer-19

  subgraph hydra-explorer 1.0.0
    explorer[hydra-explorer]
    explorer -. GET /heads .-> hydra-explorer-web
  end

  observer-19-preview -. POST /v1/observations/preview/19 .-> explorer
  observer-19 -. POST /v1/observations/mainnet/19 .-> explorer
  observer-20 -. POST /v1/observations/mainnet/20 .-> explorer

Loading

• 📊 Adds an Observer REST API, next to the Client REST API. Re-organizing code to have both web servers running on different ports. See README with instructions on how to run it.
• 📊 Adds NetworkId and HydraVersion to explorer state aggregation. The aggregation is quite conservative and does not update head state even though potential observations would tell different. This is how the aggregation works currently (we might want to change policy). Anyhow, added property tests to ensure this.

TODO:

• /tick endpoint does not say which network the data is from. We should probably change it to a /networks endpoint or so into which we aggregate all observed networks with their latest slots and blocks? @ffakenz what do you think?

[ch1bo]

@locallycompact You moved the generators into a testlib component. What is your opinion here?

[locallycompact]

This could be deriving anyclass Generic

At the moment I don't mind anything. All options feel wrong for one reason or another, but given that two of the major dependencies of hydra both use hedgehog (cardano-ledger and cardano-api), it feels like the path of least resistance to just align there, switch to hedgehog and use named generators uniformly across all hydra projects in a sublibrary (either :gen or :testlib). Arbitrary classes, locally defined, feel great when everything is naturally generic, but when not it is inconsistent since we then need to rely on named generators.

[ch1bo]

I meant not to ask about Arbitrary vs named generators nor even QuickCheck vs hedgehog.

My question is rather.. why do generators need to be a :testlib? Doesn't it feel weird to depend on it from an application package?

[locallycompact]

Sublibraries are just another kind of way that we have of selectively ignoring information. I think generators lend themselves well to being sublibraries since a) they typically have unique dependencies themselves distinct from the rest of the code, b) they are used generally only for testing, and noise otherwise when worrying about logic.

[locallycompact]

I don't think we should engineer it such that we depend on a testlib from an application. We could never need to do that.

[ch1bo]

So your suggestion is to move the Arbitrary ExplorerState instance (for which we need this import here) into yet another hydra-explorer:testlib component?

I see now why you brought up the hedgehog and moving away from Arbitrary classes, because moving things into a testlib here would be at odds with not wanting orphan instances (https://github.com/cardano-scaling/hydra/wiki/Coding-Standards#proposal-avoid-orphan-instances)