AWS costs sky rocketing due to traffic with Artifactory/JFrog

[arnaud-flament-ripcord]

Context:
We are using ArgoCD with Artifactory JFrog deployed on AWS.

• More and more applications are managed w/ ArgoCD.
• More and more versions of charts are stored/indexed in Artifactory.

Issue:
We are noticing a lot of network traffic between those 2.
The index.yaml is transferred up to 10 times per second
As a result, our AWS costs are skyrocketing due to data transfer 400Gb per day.

Questions:
Is there any way to limit the polling to Artifactory?
Is there any to have a local proxy/cache of Artifactory locally?

Thanks a lot for help.

[jgwest]

Hi @arnaud-flament-ripcord , you may may wish to reduce the --app-resync period, which is the frequency with which Git repositories are checked for changes (including any charts that that are referenced in those). More information is available in the Argo CD docs under High Availability.

That will allow you to limit polling in general, I'm not aware of a way to limit polling to a specific location, nor of any built-in proxy caching functionality. If anyone has any tips to share here that would be great!

[alexmt]

Sounds like we have to work on a fix for that. Argo CD pulls index.yaml to resolve helm semver range to a matching version. We assumed that this is a very lightweight operation like git ls-remote but it is not correct assumption.

Short term we can introduce index.yaml caching and cherry pick it into 1.8. Creating ticket to work on it

[arnaud-flament-ripcord]

Thanks a lot @alexmt
Is there any visible ticket reference that I can use to know when it will be implemented?

[alexmt]

Thank you for reminding. Here it is: #5659