Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate oasis files is not using the pre-analysis adjusted location file for V2 runs #1123

Closed
sambles opened this issue Oct 10, 2024 · 2 comments · Fixed by #1130 or #1161
Closed
Assignees
Labels
Milestone

Comments

@sambles
Copy link
Contributor

sambles commented Oct 10, 2024

Issue Description

The Pre-analysis hook is not using the adjusted location file for V2 runs. Both files are available to write-oasis-files task but the file used is the non adjusted location.csv

As well as fixing the issue also:

  • Make sure adjusted exposure files are stored in the inputs directory (using the same naming conventions as v1 run)

Original report: The pre-analysis hook doesn't seem to work correctly with a v2 analysis. It runs according to the logs but the modified OED files don't persist. input/location.csv is the original user-supplied file and there is no input/raw_location.csv. With a v1 analysis, both files exist and location.csv is modified as expected. This is the case for Oasis 2.3.7 and 2.3.8.

@sambles sambles added the bug label Oct 10, 2024
@sambles sambles self-assigned this Oct 10, 2024
@sambles sambles changed the title The Pre-analysis hook is not using the adjusted location file for V2 runs Generate oasis files is not using the pre-analysis adjusted location file for V2 runs Oct 10, 2024
@sambles sambles moved this to In Progress in Oasis Dev Team Tasks Oct 16, 2024
@sambles
Copy link
Contributor Author

sambles commented Oct 29, 2024

Work around fix using #1130

A better longer time fix might be using a fast hashing algo to check if the file contents has changed. This is a similar problem too #1029

I think the Oasis-Data-Manage could use something like xxhash

@sambles sambles moved this from In Progress to Waiting for Review in Oasis Dev Team Tasks Oct 30, 2024
@sambles sambles moved this from Waiting for Review to Done in Oasis Dev Team Tasks Oct 31, 2024
@awsbuild awsbuild added this to the 2.3.10 milestone Oct 31, 2024
@sambles sambles moved this from Done to In Progress in Oasis Dev Team Tasks Jan 21, 2025
@sambles
Copy link
Contributor Author

sambles commented Jan 21, 2025

Hooks not working in Release branch 2.4.0 -- needs fixing

[2025-01-21 10:41:59,494: ERROR/ForkPoolWorker-16] Task pre_analysis_hook[7a1cc71e-618f-4e1d-b51b-dc74666383cd] raised unexpected: FileNotFoundError(2, 'No such file or directory')
Traceback (most recent call last):
  File "/home/worker/.local/lib/python3.12/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.12/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.12/site-packages/celery/app/autoretry.py", line 60, in run
    ret = task.retry(exc=exc, **retry_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.12/site-packages/celery/app/task.py", line 736, in retry
    raise_with_context(exc)
  File "/home/worker/.local/lib/python3.12/site-packages/celery/app/autoretry.py", line 38, in run
    return task._orig_run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/src/model_execution_worker/distributed_tasks.py", line 451, in run
    raise error
  File "/home/worker/src/model_execution_worker/distributed_tasks.py", line 447, in run
    return fn(self, params, *args, analysis_id=analysis_id, run_data_uuid=run_data_uuid, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/src/model_execution_worker/distributed_tasks.py", line 513, in pre_analysis_hook
    params['pre_loc_file'] = filestore.put(
                             ^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.12/site-packages/oasis_data_manager/filestore/backends/base.py", line 318, in put
    self.fs.put(reference, storage_location)
  File "/home/worker/.local/lib/python3.12/site-packages/fsspec/implementations/dirfs.py", line 192, in put
    return self.fs.put(
           ^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.12/site-packages/fsspec/spec.py", line 1060, in put
    self.put_file(lpath, rpath, callback=child, **kwargs)
  File "/home/worker/.local/lib/python3.12/site-packages/fsspec/implementations/local.py", line 149, in put_file
    return self.cp_file(path1, path2, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/worker/.local/lib/python3.12/site-packages/fsspec/implementations/local.py", line 127, in cp_file
    shutil.copyfile(path1, path2)
  File "/usr/lib/python3.12/shutil.py", line 262, in copyfile
    with open(dst, 'wb') as fdst:
         ^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/shared-fs/analysis-1_files-68fa9591263c49fca24b5f4190b10632/location.csv'

@sambles sambles linked a pull request Jan 31, 2025 that will close this issue
@github-project-automation github-project-automation bot moved this from In Progress to Done in Oasis Dev Team Tasks Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
2 participants