Unable to run Quilkin proxy - crashes on startup

[lyyQwQ]

What happened:
Quilkin proxy crashes immediately after startup, regardless of installation method (both Docker and Cargo install).

What you expected to happen:
Quilkin proxy should start and run normally.

How to reproduce it (as minimally and precisely as possible):

1. Using Docker:

docker run -e RUST_BACKTRACE=1 --network host us-docker.pkg.dev/quilkin/ci/quilkin:0.10.0-dev-083d425 proxy --port 28868 --to 127.0.0.1:28869

2. Using Cargo installed version:

quilkin proxy --port 7777 --to [destination_ip:port]

Anything else we need to know?:
Both installation methods result in immediate crashes.

Environment:

• Quilkin version: 0.10.0-dev (Docker), 0.9.0 (Cargo install)
• Execution environment: Both Docker container and binary (cargo install)
• Operating system: Ubuntu 20.04.6 LTS x86_64
• CPU: AMD EPYC-Milan
• Memory: 461MiB
• Kernel: 5.4.0-200-generic
• Custom filters?: No

Logs:

Docker version logs:

{"timestamp":"2024-11-14T09:12:49.471366Z","level":"INFO","fields":{"message":"Starting Quilkin","version":"0.10.0-dev","commit":"083d4255081d7525ab660afe1293b47a553fcfc4"},"target":"quilkin::cli","filename":"src/cli.rs","threadId":"ThreadId(1)"}
{"timestamp":"2024-11-14T09:12:49.471551Z","level":"INFO","fields":{"message":"Starting admin endpoint","address":"[::]:8000"},"target":"quilkin::components::admin","filename":"src/components/admin.rs","threadId":"ThreadId(1)"}
{"timestamp":"2024-11-14T09:12:49.471604Z","level":"INFO","fields":{"message":"Starting proxy","port":28868,"proxy_id":"vm1031774.cloudsilk.io"},"target":"quilkin::cli::proxy","filename":"src/cli/proxy.rs","span":{"name":"run"},"spans":[{"name":"run"}],"threadId":"ThreadId(1)"}
{"timestamp":"2024-11-14T09:12:49.471795Z","level":"ERROR","fields":{"message":"Panic has occurred. Moving to Unhealthy","panic_info":"panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.2/src/runtime/blocking/shutdown.rs:51:21:\nCannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context."},"target":"quilkin::components::admin::health","filename":"src/components/admin/health.rs","span":{"name":"run"},"spans":[{"name":"run"}],"threadId":"ThreadId(1)"}
thread 'main' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.39.2/src/runtime/blocking/shutdown.rs:51:21:
Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.

Cargo installed version logs:

2024-11-10T06:32:07.472844Z  INFO ThreadId(01) quilkin::cli: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/cli.rs: Starting Quilkin version="0.9.0"
2024-11-10T06:32:07.472924Z  INFO ThreadId(01) quilkin::components::admin: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/components/admin.rs: Starting admin endpoint address=[::]:8000
2024-11-10T06:32:07.472991Z  INFO ThreadId(01) run: quilkin::cli::proxy: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/cli/proxy.rs: Starting proxy port=7777 proxy_id="vm1031774.cloudsilk.io"
2024-11-10T06:32:07.473486Z  INFO ThreadId(07) quilkin::components::proxy::io_uring_shared: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/components/proxy/io_uring_shared.rs: io-uring loop shutdown requested
2024-11-10T06:32:07.473508Z  INFO ThreadId(01) run: quilkin::components::proxy: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/components/proxy.rs: Quilkin is ready
2024-11-10T06:32:07.473626Z  INFO ThreadId(02) quilkin::components::proxy::packet_router: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/components/proxy/packet_router.rs: pipeline reporting task closed
2024-11-10T06:32:07.473733Z  INFO ThreadId(08) quilkin::codec::qcmp: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/codec/qcmp.rs: QCMP thread was signaled to shutdown
2024-11-10T06:32:07.473800Z  INFO ThreadId(09) quilkin::net::phoenix: /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/quilkin-0.9.0/src/net/phoenix.rs: starting phoenix HTTP service addr=[::]:7600

[markmandel]

Huh weird... how did that pass CI?

[XAMPPRocky]

@markmandel I don't think we have integration tests for this path, most of the tests are focused on the more advanced setups.

[markmandel]

I would have thought

quilkin/cloudbuild.yaml

Lines 62 to 81 in a4f9db1

	- name: gcr.io/cloud-builders/docker
	dir: ./build
	entrypoint: bash
	args:
	- "-c"
	- 'timeout --signal=INT --preserve-status 5s docker run --rm -v "/workspace/examples/proxy.yaml:/etc/quilkin/quilkin.yaml" ${_REPOSITORY}quilkin:$(make version) proxy'
	id: test-quilkin-image-default-config-file
	waitFor:
	- build
	# Command line configuration
	- name: gcr.io/cloud-builders/docker
	dir: ./build
	entrypoint: bash
	args:
	- "-c"
	- 'timeout --signal=INT --preserve-status 5s docker run -v /tmp:/etc/quilkin/ --rm ${_REPOSITORY}quilkin:$(make version) proxy --to="127.0.0.1:0"'
	id: test-quilkin-image-command-line
	waitFor:
	- build

Would have caught this 🤔 (but haven't actually looked at build results yet).

[thinkong]

Any update on this? I am getting the same problem

[lyyQwQ]

Any update on this? I am getting the same problem

No, but you can try a higher version of the system. I have another server running Ubuntu 22, and everything is working fine. But I can't use that server to meet my needs.

[thinkong]

Any update on this? I am getting the same problem

No, but you can try a higher version of the system. I have another server running Ubuntu 22, and everything is working fine. But I can't use that server to meet my needs.

Oh.. I see.. guess we are going to go back to 0.8.0 for now.. since that seems to be working atm

[markmandel]

oooh! That's why it's not picked up in CI, it only happens in older linux versions.

[lyyQwQ]

oooh! That's why it's not picked up in CI, it only happens in older linux versions.

Sorry I forgot to bring up this issue here, because I posted similar messages on Discord beforehand