[Autoscaler] Make AUTOSCALER_MAX_RESOURCE_DEMAND_VECTOR_SIZE configurable

[DmitriGekhtman]

Why are these changes needed?

This change makes AUTOSCALER_MAX_RESOURCE_DEMAND_VECTOR_SIZE configurable.

Power users may wish to submit more than 1000 tasks at once and have the autoscaler respond by immediately scaling up the requisite number of nodes.

To make this happen, AUTOSCALER_MAX_RESOURCE_DEMAND_VECTOR_SIZE must be increased beyond the 1000 cap; otherwise, the demand from most tasks is ignored and upscaling is slow.

Related issue number

Limited AUTOSCALER_MAX_RESOURCE_DEMAND_VECTOR_SIZE causes the issue experienced in #45373.

This PR provides a workaround.
After merging this PR, if a user wants, say, 10k tasks to trigger quick upscaling, then the user can increase AUTOSCALER_MAX_RESOURCE_DEMAND_VECTOR_SIZE past 10k.

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.

• I've run scripts/format.sh to lint the changes in this PR.

• I've included any doc changes needed for https://docs.ray.io/en/master/.

  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.

• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/

• Testing Strategy

  • Unit tests
  • Release tests
  • This PR is not tested :(

  I tested it experimentally by increasing AUTOSCALER_MAX_RESOURCE_DEMAND_VECTOR_SIZE to 100k and submitting 10k tasks; upscaling happened smoothly.

[DmitriGekhtman]

Looks like I need to fix the format and merge master.

[DmitriGekhtman]

Test failures unrelated.

This is a fairly straightforward, low-risk change. Could someone give it a quick review?

[pcmoritz]

Thanks for updating this! Ideally all of these environment variables would be prefixed by RAY_ but that's already violated a lot in this file and maybe AUTOSCALER_ is distinctive enough. Approving it :)