Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logging disappears when worker processes in use #302

Open
jeromekelleher opened this issue Jan 16, 2025 · 2 comments
Open

Logging disappears when worker processes in use #302

jeromekelleher opened this issue Jan 16, 2025 · 2 comments
Labels
bug Something isn't working
Milestone

Comments

@jeromekelleher
Copy link
Contributor

When we run vcf2zarr encode -p0 -vv we get lots of log output, but with p1 and greater this disappears.

Related to #296

@jeromekelleher jeromekelleher added the bug Something isn't working label Jan 16, 2025
@jeromekelleher
Copy link
Contributor Author

Note that this seems to be due to us not managing logging from the subprocesses in any meaningful way. It seemed to work with some versions of Python, which must be why I missed it.

@jeromekelleher jeromekelleher added this to the 0.1.1 release milestone Jan 17, 2025
@jeromekelleher
Copy link
Contributor Author

So there's not much to be done here in the short term I think, unfortunately there's no easy workaround. The best thing to do would be to setup a custom logging handler using queues etc as demo'd in the Python docs but this is a significant engineering task.

In the short term we need to document how to work around this when having memory issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant