Inconsistent opam tree output

[josecastillolema]

I am experiencing an inconsistent opam tree output when checking dependencies for the merlin package.
Sometimes, menhir, menhirLib and menhirSdk are listed as dependencies of the merlin package:

❯ opam tree merlin
merlin.4.10-500
├── dot-merlin-reader.4.10-500 (>= 4.9)
│   ├── dune.3.10.0 (>= 2.9.0)
│   │   ├── base-threads.base
│   │   ├── base-unix.base
│   │   └── ocaml.5.0.0 (>= 4.08)
│   │       ├── ocaml-base-compiler.5.0.0 (>= 5.0.0~ & < 5.0.1~)
│   │       └── ocaml-config.3 (>= 3)
│   │           └── ocaml-base-compiler.5.0.0 (>= 5.0.0~ | os = win32) [*]
│   ├── merlin-lib.4.10-500 (>= 4.9)
│   │   ├── csexp.1.5.2 (>= 1.5.1)
│   │   │   ├── dune.3.10.0 (>= 3.4) [*]
│   │   │   └── ocaml.5.0.0 (>= 4.03.0) [*]
│   │   ├── dune.3.10.0 (>= 2.9.0) [*]
│   │   ├── menhir.20230608  <===========================================================
│   │   │   ├── dune.3.10.0 (>= 2.8.0) [*]
│   │   │   ├── menhirLib.20230608 (= version) <===========================================================
│   │   │   │   ├── dune.3.10.0 (>= 2.8.0) [*]
│   │   │   │   └── ocaml.5.0.0 (>= 4.03.0) [*]
│   │   │   ├── menhirSdk.20230608 (= version) <===========================================================
│   │   │   │   ├── dune.3.10.0 (>= 2.8.0) [*]
│   │   │   │   └── ocaml.5.0.0 (>= 4.03.0) [*]
│   │   │   └── ocaml.5.0.0 (>= 4.03.0) [*]
│   │   ├── menhirLib.20230608 [*]
│   │   ├── menhirSdk.20230608 [*]
│   │   └── ocaml.5.0.0 (>= 5.0 & < 5.1) [*]
│   ├── ocaml.5.0.0 (>= 5.0 & < 6.0) [*]
│   └── ocamlfind.1.9.6.git (>= 1.6.0)
├── dune.3.10.0 (>= 2.9.0) [*]
├── merlin-lib.4.10-500 (= version) [*]
├── ocaml.5.0.0 (>= 5.0 & < 5.1) [*]
└── yojson.2.1.0 (>= 2.0.0)
    ├── cppo.1.6.9 (build)
    │   ├── base-unix.base [*]
    │   ├── dune.3.10.0 (>= 1.10) [*]
    │   └── ocaml.5.0.0 (>= 4.02.3) [*]
    ├── dune.3.10.0 (>= 2.7) [*]
    ├── ocaml.5.0.0 (>= 4.02.3) [*]
    └── seq.base (>= 0.2.2)
        └── ocaml.5.0.0 (>= 4.07.0) [*]

While sometimes they do not:

❯ opam tree merlin
merlin.4.10-500
├── dot-merlin-reader.4.9 (>= 4.9)
│   ├── dune.3.10.0 (>= 2.9.0)
│   │   ├── base-threads.base
│   │   ├── base-unix.base
│   │   └── ocaml.5.0.0 (>= 4.08)
│   │       ├── ocaml-base-compiler.5.0.0 (>= 5.0.0~ & < 5.0.1~)
│   │       └── ocaml-config.3 (>= 3)
│   │           └── ocaml-base-compiler.5.0.0 (>= 5.0.0~ | os = win32) [*]
│   ├── merlin-lib.4.10-500 (>= 4.9)
│   │   ├── csexp.1.5.2 (>= 1.5.1)
│   │   │   ├── dune.3.10.0 (>= 3.4) [*]
│   │   │   └── ocaml.5.0.0 (>= 4.03.0) [*]
│   │   ├── dune.3.10.0 (>= 2.9.0) [*]
│   │   └── ocaml.5.0.0 (>= 5.0 & < 5.1) [*]
│   ├── ocaml.5.0.0 (>= 4.08 & < 6.0) [*]
│   └── ocamlfind.1.9.6 (>= 1.6.0)
│       └── ocaml.5.0.0 (>= 3.08.0) [*]
├── dune.3.10.0 (>= 2.9.0) [*]
├── merlin-lib.4.10-500 (= version) [*]
├── ocaml.5.0.0 (>= 5.0 & < 5.1) [*]
└── yojson.2.1.0 (>= 2.0.0)
    ├── cppo.1.6.9 (build)
    │   ├── base-unix.base [*]
    │   ├── dune.3.10.0 (>= 1.10) [*]
    │   └── ocaml.5.0.0 (>= 4.02.3) [*]
    ├── dune.3.10.0 (>= 2.7) [*]
    ├── ocaml.5.0.0 (>= 4.02.3) [*]
    └── seq.base (>= 0.2.2)
        └── ocaml.5.0.0 (>= 4.07.0) [*]

Shouldn't the output of the opam tree command be the same always?

❯ opam config report
# opam config report
# opam-version         2.2.0~alpha2
# self-upgrade         no
# system               arch=x86_64 os=linux os-distribution=fedora os-version=38
# solver               builtin-mccs+glpk
# install-criteria     -removed,-count[avoid-version,changed],-count[version-lag,request],-count[version-lag,changed],-count[missing-depexts,changed],-changed
# upgrade-criteria     -removed,-count[avoid-version,changed],-count[version-lag,solution],-count[missing-depexts,changed],-new
# jobs                 15
# repositories         1 (http) (default repo at b61304c6)
# pinned               1 (rsync)
# current-switch       5.0.0
# invariant            ocaml-base-compiler = 5.0.0 | ocaml-system = 5.0.0
# compiler-packages    ocaml-base-compiler.5.0.0, ocaml-options-vanilla.1
# ocaml:native         true
# ocaml:native-tools   true
# ocaml:native-dynlink true
# ocaml:stubsdir       /opt/opam/5.0.0/lib/ocaml/stublibs:/opt/opam/5.0.0/lib/ocaml
# ocaml:preinstalled   false
# ocaml:compiler       5.0.0

[kit-ty-kate]

what is the result of opam pin ? I can see one pinned package in your output of the config report. My guess is that you've pinned the merlin package between the two opam tree merlin commands.

[josecastillolema]

Thanks for taking a look @kit-ty-kate
You are correct, the difference happens when I pin the merlin-lib package.

Two questions if I may:

• If menhir is a dependency of merlin-lib and merlin-lib is a dependency of merlin, shouldnt menhir be listed as a dependency of merlin when the system has no pinned packages at all?

• The opam tree output has two parts:

  • First part:

    The following actions are simulated:
    === install 6 packages
      ∗ cppo              1.6.9    [required by yojson]
    ...
    

  • Second part:

    merlin.4.10-500
    ├── dot-merlin-reader.4.9 (>= 4.9)
    ...
    

  I understand that the first part is totally dependent on the sytem, which packets are installed, etc.
  Is that true for the second part also? Should I not expect the same tree (second part) always independent of the system state (installed/pinned packages, etc.) ? If the answer is no, are there a set of flags to be passed to the command that would give me a list with all the dependencies independent of the system state?

Thanks!

[rjbou]

• If menhir is a dependency of merlin-lib and merlin-lib is a dependency of merlin, shouldnt menhir be listed as a dependency of merlin when the system has no pinned packages at all?

menhir is a dev dependency of merlin-lib, so it will be displayed as a dependency only if merlin-lib is a dev package (not installed from a release tarbal: pinned, or installed via opam install --dev).

The opam tree output has two parts:

First part is package install simulation to have full information about package dependencies, to display the tree. And that simulated install is done according the current switch state (what you call system dependent).
If you want a tree that is independent from the current switch, you can use the option --no-switch.

While looking at opam tree to answer, i found 2 bugs specific to dev packages handling. They are resolved by #5687, if you want to test it.

[josecastillolema]

Thanks for the clear explanation @rjbou !
I was not aware of the differentiation between normal and dev packages.
Btw, is it possible to pin a package without making it a dev package?

[rjbou]

No, if you pin it, it is considered as a dev package.

[josecastillolema]

Thanks, I am closing the issue as the reason is explained in your comment.
If I may ask, is there a way (without recurring to offline caches) to offline install some packages from its corresponding pre-dowloaded tar files?
Following with the merlin example, let's image I have downloaded merlin and all its dependencies (merlin.4.10-500 dot-merlin-reader.4.9 dune.3.10.0 merlin-lib.4.10-500 csexp.1.5.2 ocamlfind.1.9.6 yojson.2.1.0 cppo.1.6.9).
If I try in a disconnected environment:

• If I try to install them directly for the tar files:

  opam install merlin-4.10-500.tbz dune-3.10.0.tbz csexp-1.5.2.tbz yojson-2.1.0.tbz findlib-1.9.6.tar.gz v1.6.9.tar.gz
  

  This won't work because it will try to download a lot of packages not needed for merlin
• If I unzip them in a and try to pin some of them:

  for pkg in merlin dot-merlin-reader dune merlin-lib csexp ocamlfind yojson cppo ; do
     opam pin -y --no-action --assume-depexts --ignore-pin-depends $pkg-*
  done
  opam install -y merlin merlin-*
  

  This won't work because it will try to download menhir, menhirLib and menhirSdk