Scaling in AKS

[ptamilarasan]

I am looking for guidance on how scaling works when durable functions are running as a container in AKS.

In our scenario we'd like to use a TimerTrigger to start an orchestration batch function that will fan out to multiple activity functions and await all the tasks to do some aggregation. When built as a container and deployed to AKS it seems to always be limited to the single container instance. In AKS, is this supposed to scale up given enough tasks in the queue or only when running in a function app service plan?

I have a simple POC based on the DurableFunction template using the storage account backend that calls the SayHello activity 100 times. Each activity sleeps for 30 seconds and maxConcurrentActivityFunctions is the default 20, however this never triggers any scaling.

[cgillum]

There is no automatic scaling for Azure Functions when running in AKS. If you use Durable Functions on AKS and your app is allocated only to a single replica, then all activities, etc. will only run on that one replica. This is because the Functions runtime doesn't have any understanding of the compute platform it's running on, and therefore cann't directly control the scaling.

However, if you manually scale your pod to multiple replicas, Durable Functions will automatically distributed activity function executions across all available replicas. In other words, it can discover the new replicas on its own, but can't create new replicas on its own. Something external needs to allocate the new replicas.

For AKS and Kubernetes in general, we advise two things to get up and running with an auto-scaled Durable Functions solution:

1. Use the MSSQL storage provider for Durable Functions
2. Use the MSSQL KEDA scaler to automatically scale your pod based on the number of scheduled activity functions, etc. that need to be executed.

You can find some basic instructions for setting this up here: https://microsoft.github.io/durabletask-mssql/#/kubernetes

The reason we advise the MSSQL storage provider is because that's the only one that supports scaling via KEDA (we don't have a KEDA scaler for the default Azure Storage backend). You can still use the Azure Storage backend with Durable Functions on AKS, but you'll have to implement your own auto-scaling.

[cgillum]

The MSSQL KEDA scaler is generic, but the instructions I linked to show how you can configure it specifically for use with Durable Functions.

You could use the generic Azure Storage Queue scaler in KEDA. However, it's not sophisticated enough to understand all the different scaling rules that we recommend when using the Azure Storage backend for Durable Functions. If you want to go this route anyways, I suggest you configure it to monitor the work-item queue but not any of the control queues. That should probably cover 90% of use cases.

[ptamilarasan]

This specific workload is very bursty on a schedule we control, such as at the top of every hour so it seems like it would be better to scale based on the taskhub workitems queue rather than maintain enough replicas 24/7 to handle peak load. If we use the Storage queue trigger, the minimum replicas needs to be 1 so that at least one replica will run the TimerTrigger?

Are there any plans to update the Storage Queue or Service Bus scaler to better cover the durable function use cases? We are trying to avoid introducing MSSQL into this architecture as we currently only use Cosmos so having a MSSQL server for scaling functions seems a bit much.

[cgillum]

If we use the Storage queue trigger, the minimum replicas needs to be 1 so that at least one replica will run the TimerTrigger?

As far as I know, yes.

Are there any plans to update the Storage Queue or Service Bus scaler to better cover the durable function use cases?

No plans at this time.

[bbrandt]

Has anyone ever experimented with using the Event Hub scaler with Netherite? Technically possible?

[cgillum]

Pinging @sebastianburckhardt in case he's tried this or has thoughts on it.