Improve Static Asset Generation Performance

[adamchal]

Changes

Improve static asset generation performance by removing the await in the for loop. This change will allow the p-queue concurrency to limit the number of parallel tasks. Previously, the await in the for loop would cause all asset generation tasks to run serially.

Before:

for (const [originalPath, transforms] of staticImageList) {
	await generateImagesForPath(originalPath, transforms, assetsCreationPipeline, queue);
}

After:

for (const [originalPath, transforms] of staticImageList) {
	generateImagesForPath(originalPath, transforms, assetsCreationPipeline, queue);
}

Fixes #12845

Testing

No additional tests required.

Docs

Static asset generation performance should increase by os.cpus().length in most Astro projects with assets.

[changeset-bot]

🦋 Changeset detected

Latest commit: bd9728b

The changes in this PR will be included in the next version bump.

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[codspeed-hq]

CodSpeed Performance Report

Merging #12922 will not alter performance

_{Comparing adamchal:asset-generation-performance (bd9728b) with main (8b9d530)}

Summary

✅ 6 untouched benchmarks

[adamchal]

@ascorbic I’m skeptical that this change would affect any tests, but 5 CI tests are failing. A quick investigation reveals that some fixture assets are ENOENT. I’m looking for historical success with these tests but figured I’d check quickly with you first before digging deeper into the tests.

As you mentioned, because we await queue.onIdle(); the transformed asset should be finalized before any test would run. My fear is that this assumption is wrong, but that would likely be a dark tunnel of race conditions that just happens to work if we generate images more slowly (one-by-one).

[ascorbic]

Yes, this does make me concerned that it's not waiting correctly. One way to check could be to push all of the generateImagesForPath responses into an array and await Promise.all at the end. If that fixes it then it probably means that the idle wait isn't catching everything