Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add empirical model to ngiab #253

Merged
merged 3 commits into from
Dec 17, 2024
Merged

add empirical model to ngiab #253

merged 3 commits into from
Dec 17, 2024

Conversation

JoshCu
Copy link
Collaborator

@JoshCu JoshCu commented Dec 16, 2024

uses the new ngiab branch of ngen than contains the model in the /extern/ folder CIROH-UA/ngen#3 CIROH-UA/ngen@8e8fdcd

The extra index url line makes sure pytorch is installed as cpu only because the gpu packages / drivers make the image 6gb larger.

The package needs to be installed with UV after troute and ngen so that the numpy version matches.

I've tested this with some example data, the preprocessor will generate the correct configs if you add --em to it.

python -m ngiab_data_cli -i gage-10154200 --start 2010-10-01 --end 2010-11-01 -o lstm_test -sfr --em

you can also run the bmi unit tests inside the image by entering the bash shell in the container and

cd /ngen/ngen/extern/lstm/
python -m lstm

image

@benlee0423
Copy link

I tested the sample data with merge image, and it worked both serial and parallel run.

@benlee0423
Copy link

@JoshCu
Is lstm integration done with this PR?
Do you have more commits or another PR after this?

@JoshCu
Copy link
Collaborator Author

JoshCu commented Dec 17, 2024

@JoshCu
Is lstm integration done with this PR?
Do you have more commits or another PR after this?

This pr fully integrates the lstm, it was wonderfully simple thanks to how the repo is setup. I've referred to it as the empirical model after talking to the original authors because it's an older ml model that doesn't have all the latest improvements and performance of newer models. Hopefully it will make it easier to distinguish between this model and the latest lstm models being developed. It's referred to as the empirical model in the preprocessor also.

@benlee0423 benlee0423 self-requested a review December 17, 2024 18:54
@benlee0423 benlee0423 merged commit 72f1877 into main Dec 17, 2024
10 checks passed
@benlee0423 benlee0423 deleted the add_empirical_model branch December 17, 2024 18:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants