Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bottleneck in tar_outdated() with CAS #1399

Closed
wlandau opened this issue Dec 12, 2024 · 1 comment
Closed

bottleneck in tar_outdated() with CAS #1399

wlandau opened this issue Dec 12, 2024 · 1 comment
Assignees

Comments

@wlandau
Copy link
Member

wlandau commented Dec 12, 2024

With:

# _targets.R
library(targets)
repository <- tar_repository_cas_local()
list(
  tar_target(x, seq_len(1e4), repository = repository),
  tar_target(y, x, pattern = map(x), repository = repository)
)

and:

# R console
library(targets)
tar_make()
proffer::pprof(tar_outdated(callr_function = NULL))

I see:

Screenshot 2024-12-12 at 9 52 50 AM

targets could have a global cache to store the results of Base 64 decoding

@wlandau wlandau self-assigned this Dec 12, 2024
@wlandau
Copy link
Member Author

wlandau commented Jan 10, 2025

The optimizations leading up to d351ddf actually seem to have removed this bottleneck. (I bet it also helps that tar_repository_cas() now has a "list" method so that CAS methods seldom need to be called for up-to-date targets.)

M2 Mac:

Screenshot 2025-01-10 at 8 40 31 AM

RHEL9 cluster login node:

Screenshot 2025-01-10 at 9 09 25 AM

@wlandau wlandau closed this as not planned Won't fix, can't repro, duplicate, stale Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant