You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
At the moment, the GraphQL API can hang and cause a long query time under two known conditions:
A query is requested with a long delta between the startTime and endTime, so the processing of the data from S3 can take a few minutes (more?). This could be confusing to the user and from our point of view, could have performance repercussions / extra cost by pulling down a lot of S3 data. Note: Based on this pricing table, there may not be a cost associated per GB of data retrieved, but we could at least stop an unneeded request to S3 before it happens.
2.If the API is unable to connect to S3, the API can hang. May be worth returning an error back to the user saying that a connection to S3 couldn't be made.
Action Items (corresponding to the above list):
Look into ways to predict when a query may cause an overly long processing time. Having a limit on the start / end time is obvious, but I'd want to make sure this wouldn't reject any "legit" queries. If we want to get fancier, could setup a timeout?
Assuming the behavior with opentransit-state-api -> S3 is actually happening, return an error message when an S3 connection error occurs and terminate the query.
The text was updated successfully, but these errors were encountered:
For issue 2 if there are problems with the connection to S3, I think the timeout would be determined by the httpOptions.connectTimeout and httpOptions.timeout parameters to the AWS.S3 constructor documented at https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#constructor-property . Maybe it would be better to set those timeouts to ~10 seconds so that the API returns an error message in a reasonable amount of time.
At the moment, the GraphQL API can hang and cause a long query time under two known conditions:
startTime
andendTime
, so the processing of the data from S3 can take a few minutes (more?). This could be confusing to the user and from our point of view, could have performance repercussions / extra cost by pulling down a lot of S3 data.Note: Based on this pricing table, there may not be a cost associated per GB of data retrieved, but we could at least stop an unneeded request to S3 before it happens.
2.
If the API is unable to connect to S3, the API can hang. May be worth returning an error back to the user saying that a connection to S3 couldn't be made.Action Items (corresponding to the above list):
Assuming the behavior with opentransit-state-api -> S3 is actually happening, return an error message when an S3 connection error occurs and terminate the query.The text was updated successfully, but these errors were encountered: