clean up load testing ADR and add context

Co-authored-by: pluckyswan <[email protected]> Co-authored-by: saquino0827 <[email protected]> Co-authored-by: James Herr <[email protected]> Co-authored-by: Sylvie <[email protected]> Co-authored-by: jcrichlake <[email protected]>
CDCgov · Jan 7, 2025 · 6fcaad2 · 6fcaad2
1 parent fdcb850
commit 6fcaad2
Showing 1 changed file with 18 additions and 3 deletions.
diff --git a/adr/008-load-testing.md b/adr/008-load-testing.md
@@ -1,6 +1,7 @@
 # 8. Load Testing
 
-Date: 2022-12-21
+Initial date: 2022-12-21
+Updated: 2025-01-07
 
 ## Decision
 
@@ -16,11 +17,16 @@ Accepted.
 Load Testing will assist in determining the performance of a system under
 real-life load conditions, both normal and extreme.
 
+In January 2025, we added the capability to run load tests automatically on a schedule in Azure.
+In the deployed load tests, we're hitting mock ReportStream endpoints. This allows us to 1) not
+bombard ReportStream with unexpected traffic and 2) identify performance issues that are specific
+to the Intermediary.
+
 ## Impact
 
 ### Positive
 
-- **Scalability:** Locust.io is highly scalable and can simulate millions of users, making it ideal for both small-scale and large-scale load tests. 
+- **Scalability:** Locust.io is highly scalable and can simulate millions of users, making it ideal for both small-scale and large-scale load tests.
 
 
 - **Python-Based:** Writing tests in Python allows for flexibility and ease of use, especially for teams already familiar with the language. 
@@ -29,6 +35,9 @@ real-life load conditions, both normal and extreme.
 - **Cost Efficiency:** Locust.io is open-source, and doesn’t require licensing fees, which can reduce the overall cost of performance testing.
 
 
+- **Azure:** Running the load tests on a schedule in a more realistic enviornment gives us more consistent data
+
+
 ### Negative
 
 - **Limited Features:** Compared to more feature-rich tools, Locust.io might lack advanced performance monitoring or detailed reporting features. 
@@ -48,6 +57,12 @@ real-life load conditions, both normal and extreme.
 - **Resource Usage:** Running large-scale tests using Locust.io may require significant system resources, which could impact cost and infrastructure planning.
 
 
+- **Locust Future in Azure is Uncertain:** We were able to create a Locust test in Azure in November 2024, but as of January 2025, were unable to create another one. We've submitted a bug report, but if we remain unable to create Locust tests in Azure, we won't be able to expand this test setup to other environments
+
+
+- **Azure Load Testing Cannot be Terraformed:** Since Azure Load Testing resources can't be created/managed in Terraform, they must be created manually. This is more work and more error prone
+
+
 ### Related Issues
 
-- #76
+- #76, #1122