Throttling is too aggressive

[jacobthill]

I believe throttling was implemented as part of our efforts to improve site performance. I don't know if the settings are different in different environments. As the DLME data manager, I use the development environment for quality checking data transformations. Normal activity for me triggers our throttling which results in a screen that says "Throttled".

This impedes data aggregation work. Can we revisit the original intent behind this and come up with a better solution? Is this implemented in all environments or just dev? Does this have anything to do with bots? If not, humans don't typically use this site except Stanford devs and the DLME data manager.

[corylown]

I think it's likely we can close this issue since we're moving the web application on premise, but leaving it open for evaluation during the spring 2023 work cycle.

[jcoyne]

Might be rack-attack

[thatbudakguy]

deferring this until on-prem migration is mostly complete; then we can test again to see if the situation has changed

[jacobthill]

The issue persists after the migration on prem.

[thatbudakguy]

moving to "ready"; this needs more analysis to determine the root cause.

[thatbudakguy]

@jacobthill can you describe activity that reliably triggers the "throttled" message for you? right now, it looks like the only scenarios we throttle are:

• you try to login more than 20 times in one hour
• you make more than 15 searches in a 15-minute period, where:
  • there is no search term, and
  • you are requesting a page of results deeper than page 10

are these things that you do frequently?

[jacobthill]

Its always filtering via facets. When I load a data set like AUC, for example, I need to select each collection then check that the Type facets are correct. So I'm selecting "Data provider" > "Collection id" > "Type", then expanding "Type" for all collections.

[thatbudakguy]

got it, thanks for explaining. and do you only do this in stage? or are there scenarios where you would do it in prod as well?

[jacobthill]

Only in stage.

[thatbudakguy]

confirmed disabled in stage; re-enabled in prod. closing this.

[jacobthill]

I am reopening this issue because the user behavior is now changed. Some curatorial work that was previously done in stage is now done in prod–the same kind of faceting described above "So I'm selecting "Data provider" > "Collection id" > "Type", then expanding "Type" for all collections." We need to fill out the collections registry form (a spreadsheet with collection level metadata) and we will have multiple users working on this at the same time. Currently, we can only work for 5-10 minutes before we get throttled. We need to remove/adjust throttling in prod.

[jacobthill]

Closing based on #1864