Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes Issues w/ Stream Crashing Over Large Replay #996

Open
wants to merge 43 commits into
base: main
Choose a base branch
from
Open

Fixes Issues w/ Stream Crashing Over Large Replay #996

Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
2c68dad
fix: stream crashing.
l-monninger Jan 13, 2025
aa6d928
fix: Cargo.lock
l-monninger Jan 13, 2025
be23e69
fix: setting up for mocks.
l-monninger Jan 13, 2025
5f9d045
fix: refactor light node.
l-monninger Jan 13, 2025
4f8f2a3
fix: refactor stream.
l-monninger Jan 13, 2025
ff9136a
fix: reworking stream api.
l-monninger Jan 13, 2025
480bee7
fix: types.
l-monninger Jan 14, 2025
7aa181b
fix: refactor celestia provider.
l-monninger Jan 14, 2025
b4dc706
fix: celestia provider.
l-monninger Jan 14, 2025
40293f9
fix: refactor passthrough.
l-monninger Jan 14, 2025
3d4ea57
feat: merge in digest da.
l-monninger Jan 15, 2025
881b7d4
feat: light node refactor.
l-monninger Jan 15, 2025
d32fdf7
feat: accept both sequenced and passed through blobs.
l-monninger Jan 15, 2025
71c0ecf
fix: runners.
l-monninger Jan 15, 2025
a3d808a
fix: update build scripts.
l-monninger Jan 15, 2025
b708ae8
fix: remove cachix.
l-monninger Jan 15, 2025
2628c73
feat: merge disk-fifo.
l-monninger Jan 15, 2025
c008e29
fix: reintegrate inknown signers verifier.
l-monninger Jan 15, 2025
1e288a3
fix: use http1.
l-monninger Jan 15, 2025
54e27fa
debug: additional certificate logging.
l-monninger Jan 15, 2025
09af380
fix: enhance logging.
l-monninger Jan 15, 2025
6b43c13
fix: enhance logging.
l-monninger Jan 15, 2025
81b761f
fix: containers workflow.
l-monninger Jan 15, 2025
28c91f8
fix: use pr labels contains.
l-monninger Jan 15, 2025
c6a8593
debug: log event.
l-monninger Jan 15, 2025
9ff4337
debug: log containers.
l-monninger Jan 15, 2025
3969f77
fix: label filter.
l-monninger Jan 15, 2025
00adf06
docs: http1 warning.
l-monninger Jan 15, 2025
09f5c5e
fix: update default to http2.
l-monninger Jan 15, 2025
0be8313
fix: http2 support first class; http1 deprecated.
l-monninger Jan 16, 2025
39b8755
feat: switch to webpki.
l-monninger Jan 16, 2025
5ea357b
fix: connection protocol not passed through.
l-monninger Jan 16, 2025
922db07
fix: connection protocol.
l-monninger Jan 16, 2025
1913e58
fix: connection protocol.
l-monninger Jan 16, 2025
69f4d9a
add heartbeat blob on grpc connection
musitdev Jan 17, 2025
acfd786
create v1beta2.proto and set heatbeat with a bool
musitdev Jan 20, 2025
5f6de96
add some logs to see heartbeats
musitdev Jan 21, 2025
94c8e06
use a timer to generate heartbreak
musitdev Jan 22, 2025
f8594da
move heartbeat in the grpc send side
musitdev Jan 22, 2025
9dd4c62
remove all unwrap in the blob grpc stream processing
musitdev Jan 22, 2025
4026816
remove unnecessary comments
musitdev Jan 22, 2025
057e22d
Add sync DB archive file split in sub chunks to allow to upload bigge…
musitdev Jan 28, 2025
6e00b77
set db sync optional to allow genesis boot
musitdev Jan 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 20 additions & 14 deletions .github/workflows/build-push-containers-all.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,14 @@ on:

jobs:

log-event:
runs-on: ubuntu-latest
steps:
- name: Log the event
run: echo '${{ toJson(github.event) }}'

movement-celestia-da-light-node-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -78,7 +84,7 @@ jobs:
./scripts/movement/manifest movement-celestia-da-light-node

movement-celestia-appd-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -143,7 +149,7 @@ jobs:
./scripts/movement/manifest movement-celestia-appd

movement-celestia-bridge-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -208,7 +214,7 @@ jobs:
./scripts/movement/manifest movement-celestia-bridge

movement-full-node-setup-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -273,7 +279,7 @@ jobs:
./scripts/movement/manifest movement-full-node-setup

wait-for-celestia-light-node-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -338,7 +344,7 @@ jobs:
./scripts/movement/manifest wait-for-celestia-light-node

movement-full-node-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -403,7 +409,7 @@ jobs:
./scripts/movement/manifest movement-full-node

movement-faucet-service-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -468,7 +474,7 @@ jobs:
./scripts/movement/manifest movement-faucet-service

movement-tests-e2e-simple-interaction-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -533,7 +539,7 @@ jobs:
./scripts/movement/manifest movement-tests-e2e-simple-interaction

movement-indexer-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -598,7 +604,7 @@ jobs:
./scripts/movement/manifest movement-indexer

movement-tests-e2e-followers-consistent-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -664,7 +670,7 @@ jobs:


movement-util-build:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
# if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -730,7 +736,7 @@ jobs:


container-checks:
if: github.event.label.name == 'cicd:movement-containers' || github.ref == 'refs/heads/main'
if: contains(github.event.pull_request.labels.*.name, 'cicd:movement-containers') || github.ref == 'refs/heads/main' || github.event.label.name == 'cicd:movement-containers'
runs-on: buildjet-8vcpu-ubuntu-2204
needs:
- movement-celestia-da-light-node-manifest
Expand Down Expand Up @@ -786,7 +792,7 @@ jobs:
### Unchecked containers

bridge-service-build:
# if: github.event.label.name == 'cicd:bridge-containers' || github.ref == 'refs/heads/main'
if: github.event.label.name == 'cicd:bridge-containers' || github.ref == 'refs/heads/main'
permissions:
contents: read
packages: write
Expand Down Expand Up @@ -845,7 +851,7 @@ jobs:
./scripts/movement/manifest bridge-service

bridge-setup-build:
# if: github.event.label.name == 'cicd:bridge-containers' || github.ref == 'refs/heads/main'
if: github.event.label.name == 'cicd:bridge-containers' || github.ref == 'refs/heads/main'
permissions:
contents: read
packages: write
Expand Down
36 changes: 0 additions & 36 deletions .github/workflows/checks-all.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,11 +32,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main

- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run Cargo Check in nix environment
run: |
nix develop --command bash -c "cargo check --all-targets"
Expand Down Expand Up @@ -92,11 +87,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main

- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run Movement Full Node Tests Against Local ETH and Local Celestia
env:
CELESTIA_LOG_LEVEL: FATAL # adjust the log level while debugging
Expand Down Expand Up @@ -205,11 +195,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main

- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run Movement Full Node Tests Against Holesky and Local Celestia
env:
CELESTIA_LOG_LEVEL: FATAL # adjust the log level while debugging
Expand All @@ -236,11 +221,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main

- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run Movement DA Light Node tests in nix environment
# adjust the log level while debugging
run: CELESTIA_LOG_LEVEL=FATAL nix develop --command bash -c "just movement-celestia-da-light-node native build.setup.test.local -t=false"
Expand All @@ -267,12 +247,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main


- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run MCR Client Tests
run: nix develop --command bash -c "just mcr-client native build.local.test -t=false"

Expand All @@ -299,11 +273,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main

- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run Aptos Tests
run: |
nix develop --command bash -c "
Expand Down Expand Up @@ -333,11 +302,6 @@ jobs:
- name: Install Nix
uses: DeterminateSystems/nix-installer-action@main

- uses: cachix/cachix-action@v15
with:
name: movementlabs
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Run foundry tests
run: |
nix develop --command bash -c "
Expand Down
Loading
Loading