Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NFR Test Results for NGF version edge #2357

Closed
wants to merge 4 commits into from
Closed

NFR Test Results for NGF version edge #2357

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
719a00b
NFR Test Results for NGF version edge
nginx-bot Aug 8, 2024
6261a9b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 8, 2024
e0a5353
Merge branch 'main' into tests/nfr-tests-edge
lucacome Aug 9, 2024
ef9390d
Merge branch 'main' into tests/nfr-tests-edge
lucacome Aug 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 21 additions & 15 deletions tests/results/dp-perf/edge/edge-oss.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@

NGINX Plus: false

NGINX Gateway Fabric:

- Commit: 2ed7d4ae2f827623074c40653ac821b61ae72b63
- Date: 2024-08-08T21:29:44Z
- Dirty: false

GKE Cluster:

- Node count: 12
- k8s version: v1.29.6-gke.1038001
- k8s version: v1.29.6-gke.1254000
- vCPUs per node: 16
- RAM per node: 65855012Ki
- Max pods per node: 110
Expand All @@ -17,9 +23,9 @@ GKE Cluster:
## Test1: Running latte path based routing

```text
Requests [total, rate, throughput] 30000, 1000.03, 1000.01
Duration [total, attack, wait] 30s, 29.999s, 624.318µs
Latencies [min, mean, 50, 90, 95, 99, max] 496.907µs, 666.131µs, 651.971µs, 742.531µs, 778.713µs, 872.635µs, 11.772ms
Requests [total, rate, throughput] 30000, 1000.04, 1000.02
Duration [total, attack, wait] 29.999s, 29.999s, 582.398µs
Latencies [min, mean, 50, 90, 95, 99, max] 497.713µs, 646.508µs, 636.996µs, 718.685µs, 749.8µs, 837.355µs, 9.113ms
Bytes In [total, mean] 4770000, 159.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Expand All @@ -30,9 +36,9 @@ Error Set:
## Test2: Running coffee header based routing

```text
Requests [total, rate, throughput] 30000, 1000.03, 1000.00
Duration [total, attack, wait] 30s, 29.999s, 697.125µs
Latencies [min, mean, 50, 90, 95, 99, max] 520.741µs, 681.361µs, 670.45µs, 761.428µs, 795.946µs, 884.223µs, 9.626ms
Requests [total, rate, throughput] 30000, 1000.01, 999.99
Duration [total, attack, wait] 30s, 30s, 651.43µs
Latencies [min, mean, 50, 90, 95, 99, max] 532.944µs, 671.977µs, 657.662µs, 743.313µs, 774.07µs, 862.526µs, 13.558ms
Bytes In [total, mean] 4800000, 160.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Expand All @@ -43,9 +49,9 @@ Error Set:
## Test3: Running coffee query based routing

```text
Requests [total, rate, throughput] 30000, 1000.02, 999.99
Duration [total, attack, wait] 30s, 29.999s, 753.834µs
Latencies [min, mean, 50, 90, 95, 99, max] 523.903µs, 693.564µs, 681.217µs, 776.952µs, 817.06µs, 913.016µs, 6.02ms
Requests [total, rate, throughput] 30000, 1000.07, 1000.05
Duration [total, attack, wait] 29.999s, 29.998s, 619.624µs
Latencies [min, mean, 50, 90, 95, 99, max] 525.422µs, 671.622µs, 658.493µs, 749.338µs, 782.902µs, 870.726µs, 10.618ms
Bytes In [total, mean] 5040000, 168.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Expand All @@ -57,8 +63,8 @@ Error Set:

```text
Requests [total, rate, throughput] 30000, 1000.02, 1000.00
Duration [total, attack, wait] 30s, 29.999s, 680.402µs
Latencies [min, mean, 50, 90, 95, 99, max] 511.054µs, 680.638µs, 669.25µs, 763.254µs, 800.238µs, 900.815µs, 5.474ms
Duration [total, attack, wait] 30s, 29.999s, 639.035µs
Latencies [min, mean, 50, 90, 95, 99, max] 499.212µs, 665.644µs, 653.81µs, 739.852µs, 772.841µs, 860.586µs, 12.638ms
Bytes In [total, mean] 4710000, 157.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Expand All @@ -69,9 +75,9 @@ Error Set:
## Test5: Running tea POST method based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.98
Duration [total, attack, wait] 30.001s, 30s, 726.41µs
Latencies [min, mean, 50, 90, 95, 99, max] 511.128µs, 684.525µs, 671.866µs, 765.429µs, 801.43µs, 904.838µs, 10.938ms
Requests [total, rate, throughput] 30000, 1000.03, 1000.01
Duration [total, attack, wait] 30s, 29.999s, 626.728µs
Latencies [min, mean, 50, 90, 95, 99, max] 511.888µs, 661.603µs, 651.396µs, 730.352µs, 761.469µs, 839.857µs, 10.937ms
Bytes In [total, mean] 4710000, 157.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Expand Down
48 changes: 27 additions & 21 deletions tests/results/dp-perf/edge/edge-plus.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@

NGINX Plus: true

NGINX Gateway Fabric:

- Commit: 2ed7d4ae2f827623074c40653ac821b61ae72b63
- Date: 2024-08-08T21:29:44Z
- Dirty: false

GKE Cluster:

- Node count: 12
- k8s version: v1.29.6-gke.1038001
- k8s version: v1.29.6-gke.1254000
- vCPUs per node: 16
- RAM per node: 65855012Ki
- Max pods per node: 110
Expand All @@ -17,10 +23,10 @@ GKE Cluster:
## Test1: Running latte path based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.98
Duration [total, attack, wait] 30.001s, 30s, 766.8µs
Latencies [min, mean, 50, 90, 95, 99, max] 535.083µs, 732.524µs, 709.951µs, 801.487µs, 837.721µs, 978.661µs, 19.138ms
Bytes In [total, mean] 4770000, 159.00
Requests [total, rate, throughput] 30000, 1000.01, 999.99
Duration [total, attack, wait] 30s, 30s, 737.955µs
Latencies [min, mean, 50, 90, 95, 99, max] 545.611µs, 757.197µs, 734.577µs, 845.22µs, 887.95µs, 1.018ms, 12.663ms
Bytes In [total, mean] 4740000, 158.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -30,10 +36,10 @@ Error Set:
## Test2: Running coffee header based routing

```text
Requests [total, rate, throughput] 30000, 1000.02, 1000.00
Duration [total, attack, wait] 30s, 29.999s, 750.962µs
Latencies [min, mean, 50, 90, 95, 99, max] 569.894µs, 751.471µs, 737.453µs, 830.219µs, 865.522µs, 975.891µs, 13.021ms
Bytes In [total, mean] 4800000, 160.00
Requests [total, rate, throughput] 30000, 1000.01, 999.98
Duration [total, attack, wait] 30.001s, 30s, 826.722µs
Latencies [min, mean, 50, 90, 95, 99, max] 576.284µs, 794.218µs, 779.926µs, 898.695µs, 945.876µs, 1.059ms, 11.092ms
Bytes In [total, mean] 4770000, 159.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -43,23 +49,23 @@ Error Set:
## Test3: Running coffee query based routing

```text
Requests [total, rate, throughput] 30000, 1000.00, 999.98
Duration [total, attack, wait] 30.001s, 30s, 795.267µs
Latencies [min, mean, 50, 90, 95, 99, max] 566.893µs, 746.284µs, 731.685µs, 821.144µs, 856.146µs, 949.534µs, 18.228ms
Bytes In [total, mean] 5040000, 168.00
Requests [total, rate, throughput] 29999, 1000.00, 999.97
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 request is missing?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always a chance for the traffic starting a few milliseconds after intended, which I guess could lead to a missed request.

All 200s which is ultimately what we want.

Duration [total, attack, wait] 30s, 29.999s, 770.831µs
Latencies [min, mean, 50, 90, 95, 99, max] 558.235µs, 780.085µs, 765.963µs, 884.549µs, 927.74µs, 1.04ms, 10.613ms
Bytes In [total, mean] 5009833, 167.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Status Codes [code:count] 200:29999
Error Set:
```

## Test4: Running tea GET method based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.98
Duration [total, attack, wait] 30.001s, 30s, 733.608µs
Latencies [min, mean, 50, 90, 95, 99, max] 545.997µs, 736.79µs, 725.161µs, 816.5µs, 852.368µs, 950.282µs, 9.048ms
Bytes In [total, mean] 4710000, 157.00
Duration [total, attack, wait] 30.001s, 30s, 770.31µs
Latencies [min, mean, 50, 90, 95, 99, max] 564.785µs, 784.552µs, 767.705µs, 886.201µs, 931.805µs, 1.046ms, 14.459ms
Bytes In [total, mean] 4680000, 156.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -69,10 +75,10 @@ Error Set:
## Test5: Running tea POST method based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.98
Duration [total, attack, wait] 30.001s, 30s, 743.446µs
Latencies [min, mean, 50, 90, 95, 99, max] 553.99µs, 726.832µs, 715.508µs, 802.061µs, 835.171µs, 926.755µs, 8.575ms
Bytes In [total, mean] 4710000, 157.00
Requests [total, rate, throughput] 30000, 1000.02, 999.99
Duration [total, attack, wait] 30s, 29.999s, 807.239µs
Latencies [min, mean, 50, 90, 95, 99, max] 571.22µs, 768.337µs, 753.514µs, 869.559µs, 914.12µs, 1.031ms, 9.796ms
Bytes In [total, mean] 4680000, 156.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand Down
26 changes: 16 additions & 10 deletions tests/results/ngf-upgrade/edge/edge-oss.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@

NGINX Plus: false

NGINX Gateway Fabric:

- Commit: 2ed7d4ae2f827623074c40653ac821b61ae72b63
- Date: 2024-08-08T21:29:44Z
- Dirty: false

GKE Cluster:

- Node count: 12
- k8s version: v1.29.6-gke.1038001
- k8s version: v1.29.6-gke.1254000
- vCPUs per node: 16
- RAM per node: 65855012Ki
- Max pods per node: 110
Expand All @@ -18,28 +24,28 @@ GKE Cluster:

```text
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.991s, 59.99s, 799.917µs
Latencies [min, mean, 50, 90, 95, 99, max] 450.094µs, 821.717µs, 823.258µs, 951.13µs, 997.824µs, 1.126ms, 11.934ms
Bytes In [total, mean] 920025, 153.34
Duration [total, attack, wait] 59.992s, 59.991s, 903.286µs
Latencies [min, mean, 50, 90, 95, 99, max] 663.853µs, 993.737µs, 947.332µs, 1.156ms, 1.236ms, 1.5ms, 13.09ms
Bytes In [total, mean] 912000, 152.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Error Set:
```

![https.png](https.png)
![https-oss.png](https-oss.png)

## Test: Send http /coffee traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.02
Duration [total, attack, wait] 59.991s, 59.99s, 794.113µs
Latencies [min, mean, 50, 90, 95, 99, max] 618.014µs, 872.17µs, 856.784µs, 991.181µs, 1.038ms, 1.148ms, 12.075ms
Bytes In [total, mean] 960000, 160.00
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.992s, 59.991s, 966.094µs
Latencies [min, mean, 50, 90, 95, 99, max] 621.594µs, 937.441µs, 902.062µs, 1.087ms, 1.153ms, 1.345ms, 12.671ms
Bytes In [total, mean] 954000, 159.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Error Set:
```

![http.png](http.png)
![http-oss.png](http-oss.png)
24 changes: 15 additions & 9 deletions tests/results/ngf-upgrade/edge/edge-plus.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,16 @@

NGINX Plus: true

NGINX Gateway Fabric:

- Commit: 2ed7d4ae2f827623074c40653ac821b61ae72b63
- Date: 2024-08-08T21:29:44Z
- Dirty: false

GKE Cluster:

- Node count: 12
- k8s version: v1.29.6-gke.1038001
- k8s version: v1.29.6-gke.1254000
- vCPUs per node: 16
- RAM per node: 65855012Ki
- Max pods per node: 110
Expand All @@ -17,10 +23,10 @@ GKE Cluster:
## Test: Send https /tea traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.02
Duration [total, attack, wait] 59.991s, 59.99s, 867.428µs
Latencies [min, mean, 50, 90, 95, 99, max] 653.414µs, 901.844µs, 879.015µs, 1.006ms, 1.051ms, 1.18ms, 12.8ms
Bytes In [total, mean] 919993, 153.33
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.992s, 59.991s, 825.711µs
Latencies [min, mean, 50, 90, 95, 99, max] 629.57µs, 918.636µs, 874.76µs, 1.036ms, 1.098ms, 1.312ms, 12.532ms
Bytes In [total, mean] 912000, 152.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Expand All @@ -32,10 +38,10 @@ Error Set:
## Test: Send http /coffee traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.02
Duration [total, attack, wait] 59.991s, 59.99s, 839.172µs
Latencies [min, mean, 50, 90, 95, 99, max] 421.992µs, 816.859µs, 820.308µs, 942.646µs, 983.206µs, 1.111ms, 11.939ms
Bytes In [total, mean] 960000, 160.00
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.991s, 59.991s, 577.325µs
Latencies [min, mean, 50, 90, 95, 99, max] 427.641µs, 883.927µs, 851.926µs, 1.006ms, 1.068ms, 1.317ms, 12.988ms
Bytes In [total, mean] 955970, 159.33
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Expand Down
Binary file modified tests/results/ngf-upgrade/edge/http-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/ngf-upgrade/edge/http-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/ngf-upgrade/edge/https-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/ngf-upgrade/edge/https-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPRoutes/cpu-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPRoutes/cpu-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPRoutes/memory-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPRoutes/memory-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPRoutes/ttr-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPRoutes/ttr-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPSListeners/cpu-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPSListeners/cpu-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPSListeners/memory-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPSListeners/memory-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPSListeners/ttr-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_HTTPSListeners/ttr-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_Listeners/cpu-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_Listeners/cpu-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_Listeners/memory-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_Listeners/memory-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_Listeners/ttr-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_Listeners/ttr-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_UpstreamServers/cpu-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_UpstreamServers/cpu-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_UpstreamServers/memory-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/scale/edge/TestScale_UpstreamServers/memory-plus.png
View file
Open in desktop
Loading
Loading