Gemini API exhausted, need some pause mechanism or something

[VooDisss]

I'm using gemini API. You may be familar that it is free currently in experimental stage and it's very good. But there is a limit of something like 15 requests a minute and seemingly marker does not take that into account. Need implementation for this pause functionality.

ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
LLM layout relabelling: 18it [00:20,  2.63s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
LLM layout relabelling: 19it [00:24,  2.97s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
LLM layout relabelling: 21it [00:34,  3.89s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
LLM layout relabelling: 23it [00:42,  3.67s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
LLM layout relabelling: 24it [00:52,  5.39s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
LLM layout relabelling: 39it [01:05,  2.17it/s]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
LLM layout relabelling: 40it [01:05,  2.38it/s]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
LLM layout relabelling: 41it [01:06,  2.84it/s]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
LLM layout relabelling: 43it [01:16,  2.21s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
LLM layout relabelling: 44it [01:16,  1.73s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
LLM layout relabelling: 45it [01:20,  2.36s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
LLM layout relabelling: 46it [01:24,  2.59s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
LLM layout relabelling: 47it [01:34,  5.01s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
LLM layout relabelling: 48it [01:39,  4.84s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 3 seconds... (Attempt 1/3)
LLM layout relabelling: 49it [01:42,  4.33s/it]ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 6 seconds... (Attempt 2/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
ResourceExhausted: 429 Resource has been exhausted (e.g. check quota).. Retrying in 9 seconds... (Attempt 3/3)
LLM layout relabelling: 51it [01:57,  2.31s/it]
Detecting bboxes: 0it [00:00, ?it/s]
Token indices sequence length is longer than the specified maximum sequence length for this model (463 > 384). Running this sequence through the model will result in indexing errors

[clarin-ebtio800090]

Hello, we tried to solve the issue.

This is what we did:

Added rate limiting functionality to the GoogleModel class to handle the Gemini API's limit of approximately 15 requests per minute. Implemented a token bucket algorithm to manage request rates and added a wait_if_needed method to pause between requests when necessary.

You can review changes in this commit: clarin-ebtio800090@238f44d.

Caution

Disclaimer: The concept of solution was created by AI and you should never copy paste this code before you check the correctness of generated code. Solution might not be complete, you should use this code as an inspiration only.

---

Latta AI seeks to solve problems in open source projects as part of its mission to support developers around the world. Learn more about our mission at https://latta.ai/ourmission . If you no longer want Latta AI to attempt solving issues on your repository, you can block this account.