Server Side Consistency Docs (hashicorp#14392)

* partial docs

* remove unnecessary docs link

* move SSCT upgrade notes to 1.10 instead of 0.10

* Update website/content/docs/enterprise/consistency.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/enterprise/consistency.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/enterprise/consistency.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/enterprise/consistency.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/enterprise/consistency.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/enterprise/consistency.mdx

Co-authored-by: Loann Le <[email protected]>

* docs updates

* Update website/content/docs/configuration/replication.mdx

Co-authored-by: Loann Le <[email protected]>

* Update website/content/docs/configuration/replication.mdx

Co-authored-by: Loann Le <[email protected]>

Co-authored-by: Loann Le <[email protected]>

CHANGELOG.md

@@ -19,7 +19,8 @@ FEATURES:
 * **Postgres in the UI**: Postgres DB is now supported by the UI [[GH-12945](https://github.com/hashicorp/vault/pull/12945)]
 * **Remount across Namespaces (Enterprise)**: Secret engines and auth methods mounted at a namespace path are now able to be moved to a different namespace path
 * **Report in-flight requests**: Adding a trace capability to show in-flight requests, and a new gauge metric to show the total number of in-flight requests [[GH-13024](https://github.com/hashicorp/vault/pull/13024)]
-* **Server Side Consistent Tokens**: Service tokens now use SSC token format and token prefixes are updated. [[GH-14109](https://github.com/hashicorp/vault/pull/14109)]
+* **Server Side Consistent Tokens**: Service tokens have been updated to be longer (a minimum of 95 bytes) and token prefixes for all token types are updated from s., b., and r. to hvs., hvb., and hvr. for service, batch, and recovery tokens respectively. Vault clusters with integrated storage will now have read-after-write
+consistency by default. [[GH-14109](https://github.com/hashicorp/vault/pull/14109)]
 * **Transit SHA-3 Support**: Add support for SHA-3 in the Transit backend. [[GH-13367](https://github.com/hashicorp/vault/pull/13367)]
 * **Transit Time-Based Key Autorotation**: Add support for automatic, time-based key rotation to transit secrets engine, including in the UI. [[GH-13691](https://github.com/hashicorp/vault/pull/13691)]
 * **UI Client Count Improvements**: Restructures client count dashboard, making use of billing start date to improve accuracy. Adds mount-level distribution and filtering. [[GH-client-counts](https://github.com/hashicorp/vault/pull/client-counts)]

changelog/14109.txt

@@ -1,3 +1,4 @@
 ```release-note:feature
-Server Side Consistent Tokens: Service tokens now use SSC token format and token prefixes are updated."
+**Server Side Consistent Tokens**: Service tokens have been updated to be longer (a minimum of 95 bytes) and token prefixes for all token types are updated from s., b., and r. to hvs., hvb., and hvr. for service, batch, and recovery tokens respectively. Vault clusters with integrated storage will now have read-after-write
+consistency by default. [[GH-14109](https://github.com/hashicorp/vault/pull/14109)]
 ```

website/content/docs/configuration/replication.mdx

@@ -42,3 +42,5 @@ replication {
 - `best_effort_wal_wait_duration` `(duration: "2s")` - Legacy mitigation to try to prevent stale reads: when a
   write is made via RPC, i.e. the node handling the request didn't do the write itself, wait up to this much
   time to see the resulting WAL present locally before returning a response to the client.
+- `allow_forwarding_via_token` `(string: "")` - When set to `new_token`,  requests sent to non-active nodes
+  are forwarded if the node does not yet have the token information in storage. 

website/content/docs/enterprise/consistency.mdx

@@ -192,6 +192,20 @@ The option `when_inconsistent` controls how stale reads are prevented:
   `X-Vault-Inconsistent: forward-active-node` header as described above under
   Conditional Forwarding
 
+## Vault 1.10 Mitigations
+
+In Vault 1.10, the token format has changed, where service tokens now employ server side consistency.
+This means that by default, requests made 
+to nodes which cannot support read-after-write consistency due to 
+not having the necessary WAL index to check Vault tokens locally will output 
+a 412 status code. The Vault Go API automatically retries when receiving 412s, so 
+unless there is a considerable replication delay, users will experience
+read-after-write consistency. 
+
+The replication option [allow_forwarding_via_token](/docs/configuration/replication)
+can be used to enforce requests that would have returned 412s in the
+aforementioned way will be forwarded instead to the active node. 
+
 ## Client API helpers
 
 There are some new helpers in the `api` package to work with the new headers.

website/content/docs/upgrading/upgrade-to-1.10.x.mdx

@@ -31,3 +31,22 @@ Vault storage to an Etcd v3 cluster prior to upgrading to Vault 1.10.
 All storage migrations should have
 [backups](/docs/concepts/storage#backing-up-vault-s-persisted-data)
 taken prior to migration.
+
+ ### OTP Generation Process
+
+Customers passing in OTPs during the the process of generating root tokens must modify
+ the OTP generation to include an additional 2 characters before upgrading so that the 
+ OTP can be xor-ed with the encoded root token. This change was implemented as a result
+  of the change in the prefix from hvs. to s. for service tokens.
+
+ ## Token Format Change
+
+Token prefixes were updated to be more easily identifiable.
+
+* Service tokens previously started with s. now start with hvs.
+* Batch tokens previously started with b. now start with hvb.
+* Recovery tokens previously started with r. now start with hvr.
+
+Additionally, non-root service tokens are now longer than before. Previously, service tokens 
+were 26 characters; they now have a minimum of 95 characters. However, existing tokens will 
+still work.