Replies: 8 comments 15 replies
-
Version groups complicate this a lot, especially with the non-overlap check. The initial implementation would likely be a lot easier without version groups, but I suspect it'll be one of the first features asked for if this gets usage. What are the thoughts around whether it's worth the further implementation complexity? Adding some background to the transitive consistency check listed in the Extensions section: I think this is worth getting in. I proposed the base idea of Other than that, I'm really excited for this feature if approved. I think this will become the preferred way to declare dependencies on my team. |
Beta Was this translation helpful? Give feedback.
-
@zkochan I'll hold off on drafting a PR together until I get your thoughts. Just in case this is something we no longer want to do given #4475. Otherwise I'd be happy to work on this and own any maintenance after the feature merges. |
Beta Was this translation helpful? Give feedback.
-
I believe version management in a monorepo is essential, and consistent version feature in pnpm is great. I would like to write down a more lightweight proposal here: Assuming
if a, b both depends on package.json for A
package.json for B
I prefer
semantically, this configuration means the version of package 1.1 '*' should be supported, it means every package version should be consistent.
2.1 Maybe a new CLI command When 2.2 When
In a nutshell, consistent validates the version number in each
Like i said, version management is essential to a workspace. In fact, a different package version is actually intentionally. We should encourage users to use "*" in In this situation, another configuration is used to allow alternative versions.
In this way, user can declare |
Beta Was this translation helpful? Give feedback.
-
Sounds quite similar to the existing node cli tool called syncpack? |
Beta Was this translation helpful? Give feedback.
-
It's a nice featrue. I can't wait to use it. |
Beta Was this translation helpful? Give feedback.
-
Nice feature, I am looking forward! |
Beta Was this translation helpful? Give feedback.
-
is there any traction or plans to implement this or is this just in RFC stages |
Beta Was this translation helpful? Give feedback.
-
Thanks to everyone for following along. I'm going to close this discussion in favor of a new proposal at pnpm/rfcs#3 for now. The ideas translate fairly closely and accomplish the same goal. See pnpm/rfcs#1 (comment) for details of how the ideas map. |
Beta Was this translation helpful? Give feedback.
-
Edit: This idea has evolved into a feature called PNPM Templates. See discussion at pnpm/rfcs#3. Also see pnpm/rfcs#1 (comment) for how the ideas between this proposal translate to the PNPM Templates proposal.
Motivation
A common workflow in monorepos is the need to synchronize on the same version of a dependency.
For example, the
foo
andbar
packages of a monorepo may declare the same version ofreact
in theirpackage.json
files.Multiple versions of the same dependency can cause different flavors of problems.
Symbol()
are used. For example, React hooks will error if a component is rendered with a different copy of React in the same app.@types
package causes compile errors from mismatching definitions. The compiler diagnostic for this is usually: "argument of typeFoo
is not assignable to parameter of typeFoo
". For developers that have seen this before, they may realize this diagnostic is due to a dependency version mismatch. For developers newer to TypeScript, "Foo
is not assignable toFoo
" is very confusing.While there are situations making differing versions become unavoidable, this is usually accidental. Multiple differing versions arise from not reviewing
pnpm-lock.yaml
file changes or not searching for existing dependency specifiers before adding a new one. The later is typically unwritten convention in most monorepos.Summary
Instead of every workspace package declaring a version range on a dependency, a new
workspace-consistent
version specifier protocol would allow packages to delegate to a range defined at the rootpackage.json
.Minimal Example
A sample root `package.json` configuration would appear as such:
Workspace packages can then refer to the configured root defined version ranges:
The resulting `pnpm-lock.yaml` file would persist as:
Peers Suffix Elaboration
It's worth elaborating on the lockfile resolved version for
react-dom
, which was recorded as:workspace-consistent_react@workspace-consistent
for the minimal example above.The
react@workspace-consistent
portion (after the underscore) is referred to as the "peers suffix" in pnpm nomenclature. A "peers suffix" appears in this case sincereact-dom
declares a peer dependency onreact
.Without workspace consistent versions, this would normally render as
[email protected]
. The17.0.2
portions are intentionally replaced withworkspace-consistent
to keep changes to the lockfile small. The extra indirection allows edits to be limited to theworkspaceConsistent
andpackages
section whenever the workspace changes itsreact
orreact-dom
version. See Merge conflicts inpnpm-lock.yaml
are reduced for the problem this solves.Rationale for First-Class Support
While there's existing tooling in the frontend ecosystem for consistent versions (
syncpack
being one great option), builtin support from pnpm allows several improvements not otherwise possible.1. Merge conflicts in
pnpm-lock.yaml
are reducedWhen upgrading a dependency intended to be consistent across workspace packages, any blocks under the
importers
key containing that dependency will have line changes inpnpm-lock.yaml
.Suppose a commit upgrades
react
andreact-dom
to^18.0.0-rc.3
in the minimal example withoutworkspace-consistent
usage. This results in git merge conflicts if another commit:Changes the version of a dependency line-adjacent to the upgraded dependency.
Suppose the current
HEAD
commit upgradesjest
. Ongit merge
, the following conflict manifests.Add or removes a dependency line-adjacent to the upgraded dependency.
Suppose the current
HEAD
commit deletesjest
frombar
. Ongit merge
, the following conflict manifests.A similar merge conflict arises if
redux
was added since it would be declared belowreact
.Adds a new declaration of the upgraded dependency.
Suppose the current
HEAD
commit adds areact-dom
declaration tofoo
:As monorepos grow in workspace package count, merge conflicts become increasingly more probable. On GitHub, any merge conflict in
pnpm-lock.yaml
blocks pull requests due to lack of custom merge driver support.With first-class support for workspace consistent versions, edits on upgrades are limited to the root
package.json
and theworkspaceConsistent
portion ofpnpm-lock.yaml
. This reduces churn inpnpm-lock.yaml
, and prevents merge conflicts in all the cases listed above.Previous discussion: #4324 (comment)
2. Consistent versions can be declared directly in
package.json
files.Existing tooling rely on a synchronization step that edits all
package.json
files on a dependency upgrade. For example, developers may find + replace"react": "^17.0.1"
to"react": "^17.0.2"
across a repository.package.json
files and merge conflicts with other commits editingpackage.json
files.3. Intention becomes clear
There might be a tight relationship between
foo
andbar
.A developer working primarily in
@monorepo/bar
may not realize the implied coupling and upgrade@monorepo/bar
toreact@18
without realizing an edit to@monorepo/foo
was also required.The
workspace-consistent
protocol makes it more clear from just readingpackage.json
when a dependency is intended to be consistent across the monorepo. Ideally this person would search "workspace-consistent package.json` online and find pnpm.io docs.Implementation
The primary implementation changes happen in
pnpm-lock.yaml
read and write.On read, any
workspace-consistent
references will be replaced with the aliased version when deserializing the lockfile into memory. On write, the in-memory versions would be replaced withworkspace-consistent
.This process may not be possible if the lockfile in a broken state.
workspace-consistent
is specified for a dependency in the lockfile not present in the rootpackage.json
, installation will abort.workspace-consistent
resolution is specified for a dependency not present in theworkspaceConsistent
lockfile block, a best case resolution will be made following existing semver range to concrete version logic.Other Changes
Similar to the
workspace:
protocol,pnpm publish
will need to dynamically replace instances ofworkspace-consistent
with the value specified in the rootpackage.json
.There may be changes to
pnpm update
to make sure it respectsworkspaceConistent
versions, but the existing version deduplication logic may do that already.Elaboration
Semver Range Specifier in Workspaces
Readers familiar with the
workspace:
protocol may be curious as to why theworkspace-consistent:
protocol does not allow a semver range specifier for publishing like theworkspace:
protocol does. This is because the behavior on publish may be confusing.Walking through an example of the confusing behavior:
Assume the
react
specifier^17.0.0
resolved to17.0.2
.Suppose semver ranges were indeed allowed on
workspace-consistent:
. If a package were to specifyworkspace-consistent:^
:On publish this is expected to become:
It would be surprising if this became
^17.0.2
(created by adding^
to the resolved version) since that may be overly specific and leave authors with no way to relax that.On the other hand, suppose a package were to specify
workspace-consistent:*
:On publish, the least suprising behavior would be for this to become
17.0.2
. This is because17.0.2
was the resolved version actually tested on in the monorepo.This leads to a discrepancy between what
workspace-consistent:*
andworkspace-consistent:^
should append semver range types to on publish. (The original specifier vs the resolved version.)The semantics may become more clear if
workspaceConsistent
configured versions must be a concrete version, but this eliminates the ability topnpm update
workspaceConsistent
values.Lockfile
workspaceConsistent
block formatThe proposed format is:
An alternative format that represents the same information would be:
The alternative format better matches the existing shape of
importers
. However the proposed format is more resilient to merge conflicts. Separate commits changing the version ofreact
andjest
would merge conflict with the alternative format, while the proposed format would merge those changes cleanly.Extensions
Transitive Consistency
Multiple versions of a package can still end up in the final product due to versions declared as a dependency of a dependency.
It would be useful to declare two extra fields to solve the duplicate dependency problem completely.
enforceConsistencyTransitively
will error if there is ever more than one item in thepackages
block ofpnpm-lock.yaml
for a dependency, even duplicates on the same version but appearing multiple times due to peer dependencies.The
allowTransitiveMismatches
option will provide an escape hatch in case specific dependencies are acceptable. In the case above, duplicate@types/react
dependencies may be acceptable since it's usually underdevDependencies
and package authors may declare a semver range without any overlap on the monorepo workspace consistent range.Allowed Deviations
The default behavior is to show an error if a workspace package does not use
workspace-consistent
for a configured dependency. However, sometimes a workspace package may not be able to use the same version of a dependency as the other packages in the monorepo. TheallowedDeviations
option can be used as an escape hatch.Once specified,
@monorepo/baz
will be allowed to declare a version range that does not begin withworkspace-consistent
for@types/react
.Version Groups
Users may want to declare different subset partitions of consistent version sharing. Borrowing the phrase version groups from syncpack, this could be defined as such:
Workspace packages would then use
workspace-consistent:<version-group>
as the protocol. Simplyworkspace-consistent
would refer to thedefault
group.An edge case arises if a package overlaps its version group usage.
These scenarios are likely mistakes. While installation can continue without problems, an error should be emitted to be helpful. To produce the error a check algorithm would record usages of a non-default version group and the dependencies declared inside of it. If another version group is selected for a dependency present in a previously declared group, installation will fail.
Note that the algorithm above would permit this alternative scenario, which is likely not a problem.
Alternatives
Comparison to overrides/resolutions
An alternative mechanism for declaring workspace consistent versions is the
pnpm.overrides
feature. While mechanically this allows you to set the version of a dependency across all workspace packages, it can be a bit unexpected when ifpnpm.overrides
rewrites a dependency's dependency to an incompatible version silently.pnpm.overrides
is ultimately intended for a different purpose. The NPM RFC for a similar feature explicitly states that it should be used as a short-term hack to fix vendor problems.https://github.com/npm/rfcs/blob/main/accepted/0036-overrides.md
The
workspace-consistent
protocol is conversely intended for long-lived usage.Beta Was this translation helpful? Give feedback.
All reactions