Add Swift 6

[finestructure]

Add a new Swift version (major or minor)

This guide assumes that an older Swift version is removed when adding a new one. This makes the process less cumbersome, because updating the test expectations in the Server package (of which there are a number) is easier if one just shuffles Swift version numbers around without having to add and remove versions separately.

Steps marked with ⚠️ are the ones that are relevant for deployment and need to be held back until everything is in place.

• Open up a tracking issue with the contents of this guide

Example issue: #2890

Update SPIManifest

Preconditions: None.

This can be done at any time. It only takes effect once the adopting packages (Server and Builder) update to this version.

• Update SPIManifest Drop 5.7 SPIManifest#44
• Tag new release

Example change: https://github.com/SwiftPackageIndex/SPIManifest/pull/39/files

Update PackageList

Preconditions:

• new Swift docker image available (alternatively, use a nightly toolchain)

• Update PackageList Use Swift 6 nightly for package dump PackageList#7208

• Merge

Note: There's little point in merging this in order to be able to add new packages that have their tools-version set to this new version unless the server changes have been deployed. If the server can't "package dump" this tools version the analysis step will fail and the package will not show up in the index. However, it is safe to merge this ahead of time. Packages that have been added will start appearing once the server has been updated.

Example change: https://github.com/SwiftPackageIndex/PackageList/pull/6352/files

Update PackageList-Validator

Preconditions:

• new Swift docker image available (automatically ensured by package CI)

This ensures via the CI build that the validator binary which is built in the PackageList repository can be built successfully with the latest Swift version.

• Update PackageList-Validator Use Swift 6 nightly for validation PackageList-Validator#66
• Merge

Example change: https://github.com/SwiftPackageIndex/PackageList-Validator/pull/61/files

Update spi-images

Preconditions: new Swift docker image available (automatically ensured by package CI)

The images don't need to be tagged, as they are automatically picked up via their .../spi-images:basic-5.8-latest tag.

• Update the Swift versions in the image build matrix of spi-images
  • We already have a Swift 6 image in place that we've been using for the preview runs (it just needs updating to the latest nightly next week)
• Merge

Example change: https://gitlab.com/finestructure/spi-images/-/merge_requests/19/diffs

Update builder machines

Preconditions: new Xcode version available

• Install the Xcode version on all eligible macOS builders
• Set a runner tag for the new version swift-x.y in builder CI/DC

Update Builder

Preconditions:

• step Update builder machines

• step Update spi-images

• Update to new SPIManifest version and address all issues arising from non-exhaustive switch statements.

• Update Makefile to build and test with new Xcode version (if Swift version is a release version)

• Update .gitlab-ci-builder.yml: add/remove build <platform> <swift-version> blocks.

• Update .gitlab-ci-testmatrix.yml: add/remove new test <platform> <swift-version> jobs

• ⚠️ Tag & make commit when ready to deploy https://gitlab.com/finestructure/swiftpackageindex-builder/-/merge_requests/287

• ⚠️ Merge when ready to deploy

Example change: https://gitlab.com/finestructure/swiftpackageindex-builder/-/merge_requests/253/diffs

Update spi-base

Preconditions: new Swift docker image available (automatically ensured by package CI)

• Update spi-base
• Test SPI-Server with new base image
• Merge https://gitlab.com/finestructure/spi-base/-/merge_requests/27
• Tag new version

Example change: https://gitlab.com/finestructure/spi-base/-/merge_requests/24/diffs

Update Server

Preconditions:

• step Update Builder

• step Update spi-base

• Update spi-base to newly tagged version Swift 6 nightly base image #3120

• Update SPIManifest dependency Update to SPIManifest 1.5.0 + related changes #3121

• Update SwiftVersion+Build.swift and adjust to changes

• Update PackageShow.Model.SwiftVersionResults and adjust to changes (outdated)

• Udate SwiftVersion.note in SwiftVersion+BuildResultParameter if needed

• Ensure any new SupportedPlatform keys are added to our matching Manifest.Platform.Name enum so we don’t fail to decode package manifests that use them. (See issue 1441 for details.)

• Merge Update to SPIManifest 1.5.0 + related changes #3121

• set BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING to 0.8

• ⚠️ Deploy when ready

Example change: https://github.com/SwiftPackageIndex/SwiftPackageIndex-Server/pull/2899/files

Delete old Swift version builds

Preconditions: New server (one that doesn't trigger the Swift version to be deleted) has been deployed

• ⚠️ Delete outdated builds when ready
  • dev
  • prod

Example SQL:

delete
from builds
where
swift_version->'major' = '5'
and swift_version->'minor' = '6'

This should be run once the new Server has been deployed. Running it too early will lead to the deleted versions being triggered again. It's harmless but wastes build resources.

Deleting the records too long after deployment will inhibit triggering of the new Swift versions, because the presence of the old records will make the build result set appear to be complete. This is also harmless, as these will then be triggered once the deletion has happened.

Deployment notes

The final tasks before actual rollout should be:

• set BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING
• deploy server (by tagging)
• merge builder
• delete old Swift version builds

The steps should be followed in that order, although deviations are not critical.

Setting the BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING does not take effect until the server is redeployed.

Deploying the new server version will start triggering new Swift version builds. If the builder cannot handle them yet (because it's not been updated), they will remain in the queue and will eventually be discarded.

Updating the builder means it will stop processing the deleted Swift version and add support for a new one. So if the builder were updated first, old Swift versions would still be triggered and remain in the queue until discarded.

Neither is problematic but it's best to simply immediately merge (and therefore deploy) the new builder when the new server comes online.

Finally, deleting the old Swift versions is what really starts the reprocessing. Without this step, only new packages or versions will trigger any builds for the new Swift version.

Roll-back scenario

In case anything goes wrong, rolling back the builder and the server will bring back the previous configuration.

This will however then start re-processing of the just deleted Swift version (if that has already happened). Given that, it's probably not advisable to roll back unless there's a problem that can't be fixed in any other way, or it should be done in conjunction with a db restore to bring back the deleted Swift versions.

Post deployment

Preconditions: all the above steps have been completed, not time critical

• Remove obsolete runner tag swift-x.y in builder CI/DC
• Reset BUILD_TRIGGER_LATEST_SWIFT_VERSION_DOWNSCALING to 1

[finestructure]

We're running Swift 6 builds in productions now.