Initial proposal for short-term release process

0001-short-term-release-process/0001-release-process-short-term.md

@@ -0,0 +1,253 @@
+# Short-Term Release Process for Autonomi
+
+- Status: proposed
+- Type: enhancement
+- Related components:
+- Start Date: 2024-07-08
+- Discussion:
+- Supersedes:
+- Superseded by:
+
+## Summary
+
+This document intends to provide a detailed development and release process we can follow until the
+initial launch of the network in October/November. It will be based on short release cycles that
+will last about three weeks. After launch, the cycles would likely be at least double that length
+and could be quite different in detail.
+
+It should be clear that we deal here with processes for releasing code, but we don't go into any
+detail on how to deploy it. There are issues there we will also have to address, but those will be
+for another document.
+
+## Conventions
+
+Uncertainties are indicated using a `*` notation. Rather than have a full `Unresolved Questions`
+section at the end, where applicable there is an `Uncertainties` list in each section.
+
+## Motivations
+
+So far, we have failed to arrive at a well-defined process for releasing our code. With launch
+imminent, it is my opinion that we've now run out of road for experimentation and we need to settle
+on an unambiguous process that we can execute without deliberation. Finalising this document should
+be a collaborative process in which we address any uncertainties rather than leaving them open for
+further experiment.
+
+## Detailed Design
+
+## General Branching/Merging Techniques
+
+The chosen branching and release model is closely correlated with Gitflow. Gitflow is a mature model
+that has proven useful for projects that cannot operate using continuous delivery. [This](https://nvie.com/posts/a-successful-git-branching-model/) article originally defined the model. Reading it will help you familiarise yourself with the general concepts.
+
+In our setup, we would have two permanent branches, `main` and `stable`, where `main` is for
+day-to-day development and `stable` is essentially for tracking releases we consider deployable.
+Like Gitflow, we would also have some temporary branches; these will be discussed in more detail
+later.
+
+### Merging
+
+Historically, our team has preferred `git rebase` to `git merge`; however, Gitflow is better
+supported by using `git merge`. Rebasing is fine before submitting upstream, but Gitflow involves
+merging commits between different branches. Due to the fact that rebasing rewrites the commit
+history, merging between branches can lead to commits that have the same content but different
+hashes. This can make subsequent, post-rebase merging more confusing than it needs to be and also
+cause the commit history to be littered with duplicates. The primary option for completing a PR
+should be to merge it in rather than rebase.
+
+## Release Cycle Overview
+
+This section provides an overview of a release cycle. Certain aspects here merit more detail, but
+those will be provided in subsequent sections.
+
+The cycle has the following phases and steps:
+
+* Continual internal development and testing:
+    - Feature branches are worked on and merged back into `main`
+    - Developers can deploy their own isolated testnets if necessary
+* Release candidate (RC) phase (two weeks):
+    - Twice a month, roughly corresponding two every two weeks, create a `release-YYYY.MM.rc.1`
+      branch from `main`. No new features will be accepted on this branch.
+    - Bump version numbers, with an `rc.1` suffix applied.
+    - Build RC and release to Github as a public `pre-release` but with no published crates.
+    - Deploy RC to `STG-01`.
+    - Deploy previous stable release to `STG-02`.
+    - Production of changelog begins.
+    - Invite community members to test against this network.
+    - Initiate comparisons to the previous stable release in `STG-02`.
+    - Fixes can be applied to the release branch, resulting in an `rc.2`. This will be
+      released/deployed/tested. Repeat if necessary.
+* Release and deploy phase (one/two days):
+    - If the RC passes two weeks of QA, the release branch is ready to be merged back to `stable`.
+      On that branch, modify version numbers to remove the `rc` pre-release specifier.
+    - Changelog finalised.
+    - Merge the release branch into `stable`.
+    - Merge the release branch into `main`.
+    - When ready, use a GHA workflow to perform a stable release.
+    - Delete the release branch.
+    - Deploy the new release to nodes being hosted by Maidsafe.
+    - Announce to users.
+
+We can accommodate hotfixes at any point during the cycle.
+
+## Release Cycle Anatomy
+
+We'll now elaborate the release cycle described in the last section, discussing each type of release
+in more detail.
+
+### Artifacts
+
+Our releases currently produce eight binary artifacts, so we'll discuss these first.
+
+In Rust, crates must use Semantic Versioning. A binary is defined within a crate, and therefore, by
+default, it will also have a Semantic Version; however, it is possible to override the `--version`
+argument to provide something custom. It would be useful if we could refer to these collectively
+with a single version number and package, where the package name would reflect the version number.
+The collective version number will be `YYYY.MM.X.Y`, where `X` is for the release cycle within the
+month, and `Y` is a counter that will increment for each RC branch produced within the cycle.
+
+The collective version number is used for a single Github Release. The assets for the release are
+the combined binary packages for each platform. The changelog can also be nicely applied to this
+combined release.
+
+To accommodate these and some other things, our binaries will now have several version arguments,
+which are defined as follows:
+
+* `--package-version`: outputs the collective `YYYY.MM.X.Y` version
+* `--crate-version`: outputs the crate's Semantic Version
+* `--protocol-version`: outputs the network protocol version (correlated with the Semantic Version)
+* `--version`: outputs all of the above
+
+### Alpha Releases
+
+An alpha release would accommodate a scenario in which we wanted to put out quick, experimental code
+to have community users test something on an isolated network. We can branch it off `main` and
+discard the branch when it's done. The branch is intended to have a very short duration. It will be
+possible to apply fixes to it and do a new release on the same branch, in which case the pre-version
+specifier will be incremented. If the fixes are good, we can cherry pick the fix commits back into
+`main`.
+
+An owner should be designated to the whole experiment. They will produce the release, deploy it, and
+coordinate with users.
+
+#### Process
+
+* Prepare a light description that communicates the purpose of the release; we do not need a full
+  changelog for this type of release.
+* Create and checkout an `YYYY-MM-DD-alpha.1` branch from `main`.
+* Use `release-plz update` to bump crate versions.
+* Use a script to apply the `alpha` pre-release specifier to each bumped crate.
+* Create a `chore(release): alpha-YYYY-MM-DD` commit.
+* Push that branch to the upstream `maidsafe` repo, which will kick off the release workflow.
+* The release workflow will produce a public `pre-release` on Github, but the crates will *not* be
+  published.
+* Manually edit the Github Release to provide the description prepared in the first step.
+* Use the `Launch Network` workflow to deploy the `alpha` binaries to an isolated network.
+* Announce the availability of the binaries to the community. Users can use `safenode-manager`
+  and/or `safeup` with `--version` arguments to obtain the alpha binaries.
+* Perform testing
+* If a problem is identified, there is an opportunity for a small fix/test cycle:
+    - On the same alpha branch, apply the fix.
+    - Use `cargo release version alpha --package X --package Y` to increment the necessary crates to
+      `alpha.2`
+    - Deploy the fix either using an upgrade or by launching a new testnet
+    - Users can test
+    - Repeat if necessary (in practice we should not have many of these)
+* If need be, any fix commits applied on the alpha branch should be cherry picked into `main`.
+* The experiment is over and the branch should be deleted.
+
+The branch is discarded because we don't want the alpha version bumps back in `main`. Crates were
+also not published. A Github Release always creates a tag, so this will function as the historical
+record of the existence of the alpha release.
+
+### Release Candidates
+
+A release candidate (RC) is the binary that's intended to be released as a stable version. The set
+of features and fixes in the RC is what's included on `main` in the current cycle, i.e., between now
+and the last stable release. The release candidate branch will be cut twice a month, roughly every
+two weeks. Community users will be invited to participate in testing.
+
+Once the RC branch is started, we won't accept new features on it, only fixes. Feature development
+can continue on `main`.
+
+#### Process
+
+* Create and checkout an `YYYY-MM-DD-rc.1` branch from `main`.
+* Use `release-plz update` to bump version numbers. These new versions should be the ones used for
+  the stable release that will be based on this RC branch.
+* Use a custom script to apply `rc.1` to the new versions. We can't use `cargo release` for this
+  because it also performs a `PATCH` bump when you apply the pre-release specifier, which is very
+  annoying.
+* Create a new template entry in the changelog. This can be filled out as an on-going process
+  between now and the stable release.
+* Create a `chore(release): YYYY-MM-DD-rc.1` with the version bump and changelog and push the branch
+  to `origin` or `upstream`.
+* That push should trigger a workflow that will:
+    - Build the RC
+    - Upload the binaries to S3
+    - Produce a public `pre-release` Github Release 
+    - Crates will NOT be published
+* Use a script to produce a list of the commits between now and the last stable version. The commits
+  will be grouped by author. This list can be posted in Slack or Discourse to aid developers in
+  supplying their contributions for the changelog.
+* Use an `Upgrade Network` workflow to deploy the RC to `STG-01`. This will function as a test of
+  the upgrade process and help identify breaking changes we may have missed.
+* Invite users to participate in testing the RC. They can use `safenode-manager` to obtain the `rc`
+  nodes and `safeup` for `rc` clients.
+* We can also perform our own QA testing, some of which will come from the metrics that result from
+  the comparison to the previous stable release.
+* If fixes are necessary, they should be applied to this branch. We then bump to `rc.2` and do
+  another release, which again should be deployed to `STG-01`. Users can get the new binaries. This
+  part could potentially be repeated, but obviously we want to avoid that. When the fix is verified,
+  that commit should be cherry picked back in to the `main` branch.
+
+We would now be in a position where we'd be looking to do a stable release and deploy to production.
+
+### Stable Release
+
+At this point, the release branch still exists. This process is about making a stable release from
+that branch.
+
+#### Process
+
+* Still on the release branch, use a script to remove the `rc` pre-release identifier from the
+  crates that were bumped.
+* If it isn't already, the changelog should now be finalised.
+* Create a `chore(release): YYYY.MM.X.Y` commit. Put the crate name and version bumps in the body of
+  the commit. Any final additions to the changelog can be part of this commit.
+* Create a PR for merging the release branch into `stable`.
+* Once it's been merged to stable, also merge the release branch into `main`.
+* When ready, kick off a workflow for the stable release. The workflow will:
+    - Build the binaries
+    - Upload them to S3
+    - Public Github Release
+    - Publish crates
+    - Tag based on combined version
+* Manually edit the Github Release to apply the latest changelog entry to the description
+* Delete the release branch
+
+We would now be in a position to deploy the stable release to the `PROD-01` (and possibly `PROD-02`)
+environment. The production deployment would be covered in another RFC.
+
+## Hotfixes
+
+Hotfixes are intended to quickly fix a severe bug in a stable release. They can occur at any time
+throughout the release cycle, although they'd probably more likely be near the beginning.
+
+### Process
+
+* Create and checkout a `hotfix-YYYY.MM.DD` branch from `stable` and push it to `origin`.
+* Create an entry in the changelog that describes the fix.
+* Use `release-plz update` to get new version numbers for the crates that the fix applies to.
+* Use a script to apply an `rc.1` pre-release specifier to the bumped crates.
+* Create a `chore(release): hotfix-YYYY.MM.DD` commit with the bumped crates and versions in the
+  body of the commit.
+* Fetch this branch from `upstream` to a fork and apply the fix.
+* PR the commit with the fix to the `upstream` branch. This will enable someone to review it.
+* If changes are requested, keep going until those are resolved.
+* Use a workflow to deploy the fix to a dev/staging environment to be tested.
+* When the fix is confirmed to be working:
+    - Remove the `rc` pre-release specifier
+    - Create a PR to merge the branch back into `stable`
+* Also merge it back into `main`.
+* Perform a stable release at the new version number.
+* If it's a change to the node, deploy it to production using an upgrade process.