Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix JLL version fix fix #4130

Merged
merged 6 commits into from
Jan 10, 2025
Merged

Conversation

IanButterworth
Copy link
Member

Reported here JuliaRegistries/General#122129 (comment)

ERROR: KeyError: key v"1.0.11+3" not found
Stacktrace:
  [1] getindex
    @ ./dict.jl:498 [inlined]
  [2] resolve_versions!(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, julia_version::VersionNumber, installed_only::Bool)
    @ Pkg.Operations ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:452
  [3] up(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, level::Pkg.Types.UpgradeLevel; skip_writing_project::Bool, preserve::Nothing)
    @ Pkg.Operations ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1546
  [4] up
    @ ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/Operations.jl:1528 [inlined]
  [5] up(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; level::Pkg.Types.UpgradeLevel, mode::Pkg.Types.PackageMode, preserve::Nothing, update_registry::Bool, skip_writing_project::Bool, kwargs::@Kwargs{})
    @ Pkg.API ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/API.jl:351
  [6] up
    @ ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/API.jl:326 [inlined]
  [7] up
    @ ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/API.jl:164 [inlined]
  [8] #resolve#143
    @ ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/API.jl:357 [inlined]
  [9] resolve
    @ ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/API.jl:356 [inlined]
 [10] resolve(; io::Base.TTY, kwargs::@Kwargs{})
    @ Pkg.API ~/.asdf/installs/julia/1.10.7/share/julia/stdlib/v1.10/Pkg/src/API.jl:355
 [11] top-level scope
    @ none:1

@IanButterworth
Copy link
Member Author

I don't understand why currently but this bug only happens when the JLLs are in the sysimage

vers_fix[uuid] = old_v
new_v != old_v && delete!(compat_map[uuid], new_v)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this shouldn't be deleted, for cases where the package version in Project.toml and the manifest don't match, and the package is in the sysimage. i.e. #4131

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@KristofferC do you think this delete! should be here? I'm leaning towards no to be safer in case I haven't thought through all situations, but not sure

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed it with a comment

@IanButterworth
Copy link
Member Author

Tested and appears to work.
With a sysimage with Xorg_libXau_jll v1.0.11+3 in it (which has a version in its Project.toml of v1.0.11+1)

Master

% ./julia --sysimage=JLLTestSysimage.so --project=jll_test
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.0-DEV.1869 (2025-01-10)
 _/ |\__'_|_|_|\__'_|  |  Commit d3964b600a (0 days old master)
|__/                   |

(jll_test) pkg> st
Status `~/Documents/GitHub/julia/jll_test/Project.toml`
  [0c0b7dd1] Xorg_libXau_jll v1.0.11+3

(jll_test) pkg> add Example
   Resolving package versions...
ERROR: KeyError: key v"1.0.11+3" not found
Stacktrace:
  [1] getindex
    @ ./dict.jl:477 [inlined]
  [2] resolve_versions!(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, julia_version::VersionNumber, installed_only::Bool)
    @ Pkg.Operations ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:539
  [3] targeted_resolve(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, preserve::Pkg.Types.PreserveLevel, julia_version::VersionNumber)
    @ Pkg.Operations ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:1607
  [4] tiered_resolve(env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, julia_version::VersionNumber, try_all_installed::Bool)
    @ Pkg.Operations ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:1572
  [5] _resolve(io::IOContext{IO}, env::Pkg.Types.EnvCache, registries::Vector{Pkg.Registry.RegistryInstance}, pkgs::Vector{Pkg.Types.PackageSpec}, preserve::Pkg.Types.PreserveLevel, julia_version::VersionNumber)
    @ Pkg.Operations ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:1615
  [6] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, new_git::Set{Base.UUID}; allow_autoprecomp::Bool, preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, target::Symbol)
    @ Pkg.Operations ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:1651
  [7] add
    @ ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/Operations.jl:1623 [inlined]
  [8] add(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; preserve::Pkg.Types.PreserveLevel, platform::Base.BinaryPlatforms.Platform, target::Symbol, allow_autoprecomp::Bool, kwargs::@Kwargs{io::IOContext{IO}})
    @ Pkg.API ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/API.jl:317
  [9] add(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{})
    @ Pkg.API ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/API.jl:164
 [10] add(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/API.jl:153
 [11] do_cmd(command::Pkg.REPLMode.Command, io::Base.TTY)
    @ Pkg.REPLMode ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/REPLMode/REPLMode.jl:409
 [12] do_cmds(commands::Vector{Pkg.REPLMode.Command}, io::Base.TTY)
    @ Pkg.REPLMode ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/src/REPLMode/REPLMode.jl:395
 [13] do_cmds(repl::REPL.LineEditREPL, commands::String)
    @ REPLExt ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/ext/REPLExt/REPLExt.jl:99
 [14] on_done(s::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool, repl::REPL.LineEditREPL)
    @ REPLExt ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/ext/REPLExt/REPLExt.jl:113
 [15] #invokelatest#1
    @ ./essentials.jl:1056 [inlined]
 [16] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [17] (::REPLExt.var"#create_mode##0#create_mode##1"{REPL.LineEditREPL})(s::REPL.LineEdit.MIState, buf::IOBuffer, ok::Bool)
    @ REPLExt ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/Pkg/ext/REPLExt/REPLExt.jl:135
 [18] #invokelatest#1
    @ ./essentials.jl:1056 [inlined]
 [19] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [20] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
    @ REPL.LineEdit ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/REPL/src/LineEdit.jl:2800
 [21] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
    @ REPL ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/REPL/src/REPL.jl:1681
 [22] (::REPL.var"#59#60"{REPL.LineEditREPL, REPL.REPLBackendRef})()
    @ REPL ~/Documents/GitHub/julia/usr/share/julia/stdlib/v1.12/REPL/src/REPL.jl:687

This PR

% ./julia --sysimage=JLLTestSysimage.so --project=../Pkg.jl -ie "using Pkg"
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: wrong source (1), mismatched flags (2), wrong julia version (1))
[ Info: Precompiling REPLExt [e5eb5ef1-03cf-53a7-ae1d-5a66b08e832b] (cache misses: wrong dep version loaded (1), mismatched flags (1), wrong julia version (1))
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.0-DEV.1869 (2025-01-10)
 _/ |\__'_|_|_|\__'_|  |  Commit d3964b600a (0 days old master)
|__/                   |

(Pkg) pkg> activate jll_test
  Activating project at `~/Documents/GitHub/julia/jll_test`

(jll_test) pkg> st
Status `~/Documents/GitHub/julia/jll_test/Project.toml`
  [0c0b7dd1] Xorg_libXau_jll v1.0.11+3

(jll_test) pkg> add Example
   Resolving package versions...
    Updating `~/Documents/GitHub/julia/jll_test/Project.toml`
  [7876af07] + Example v0.5.5
    Updating `~/Documents/GitHub/julia/jll_test/Manifest.toml`
  [7876af07] + Example v0.5.5
Precompiling packages finished.
  1 dependency successfully precompiled in 1 seconds

(jll_test) pkg> st
Status `~/Documents/GitHub/julia/jll_test/Project.toml`
  [7876af07] Example v0.5.5
  [0c0b7dd1] Xorg_libXau_jll v1.0.11+3

KristofferC
KristofferC previously approved these changes Jan 10, 2025
Copy link
Member

@KristofferC KristofferC left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. I wonder how hard it would be to make a test.

@IanButterworth IanButterworth marked this pull request as ready for review January 10, 2025 16:29
@IanButterworth
Copy link
Member Author

I wonder how hard it would be to make a test.

If there was a stdlib JLL that's marked as upgradable, I think we could do a test relatively easily. That's not the case, I believe?

But without that it'd require PackageCompiler which would take time and need manual installation over on julia CI because test deps aren't automatically installed there.

@IanButterworth IanButterworth added the merge me Merge when tests pass label Jan 10, 2025
@IanButterworth IanButterworth merged commit bc9fb21 into JuliaLang:master Jan 10, 2025
7 checks passed
@IanButterworth IanButterworth deleted the ib/fix_jll_fix branch January 10, 2025 17:44
IanButterworth added a commit that referenced this pull request Jan 10, 2025
(cherry picked from commit bc9fb21)
IanButterworth added a commit that referenced this pull request Jan 10, 2025
(cherry picked from commit bc9fb21)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants