RLS Attempting to access nonexistant ~/.opam/4.07.0 folder

[jameskraus]

While trying to open a very simple dune project, I get this error:

Setting log location: /Users/james.kraus/test-ocaml-program/node_modules/.lsp/debug.log
/bin/sh: /Users/james.kraus/.opam/4.07.0/bin/ocamlopt.opt: No such file or directory
/bin/sh: /Users/james.kraus/.opam/4.07.0/bin/ocamlopt.opt: No such file or directory
/bin/sh: /Users/james.kraus/.opam/4.07.0/bin/ocamlopt.opt: No such file or directory
[Error - 12:07:29 AM] Request textDocument/codeAction failed.
  Message: Could not run compiler (ran /Users/james.kraus/.opam/4.07.0/bin/ocamlopt.opt -version). Output: 
  Code: -32603 

Excerpt from the log file:

Read message 
{"jsonrpc":"2.0","method":"$/cancelRequest","params":{"id":4}}
Read message 
{"jsonrpc":"2.0","id":7,"method":"textDocument/codeAction","params":{"textDocument":{"uri":"file:///Users/james.kraus/test-ocaml-program/sum.ml"},"range":{"start":{"line":0,"character":0},"end":{"line":0,"character":0}},"context":{"diagnostics":[]}}}
[server] Got a method textDocument/codeAction
[server] processing took 0.0360012054443ms
Found a `dune` file at /Users/james.kraus/test-ocaml-program
]] Making a new jbuilder package at /Users/james.kraus/test-ocaml-program
=== Project root: /Users/james.kraus/test-ocaml-program
Detected `opam` dependency manager for local use
=== Build dir:    /Users/james.kraus/test-ocaml-program/_build
Get ocaml stdlib dirs
Include subdirs? no :/
Got a compiled base /Users/james.kraus/test-ocaml-program/_build/default/.sum.eobjs
Local file: /Users/james.kraus/test-ocaml-program/sum.ml
Local .cmt file: /Users/james.kraus/test-ocaml-program/_build/default/.sum.eobjs/Sum.cmt
>> Collecting deps for /Users/james.kraus/.opam/system/lib/stdio
>> Collecting deps for /Users/james.kraus/.opam/system/lib/sexplib0
>> Collecting deps for /Users/james.kraus/.opam/system/lib/base/shadow_stdlib
>> Collecting deps for /Users/james.kraus/.opam/system/lib/base/caml
>> Collecting deps for /Users/james.kraus/.opam/system/lib/base
>> Collecting deps for /Users/james.kraus/.opam/4.07.0/lib/ocaml
Depedency dirs /Users/james.kraus/.opam/system/lib/stdio /Users/james.kraus/.opam/system/lib/sexplib0 /Users/james.kraus/.opam/system/lib/base/shadow_stdlib /Users/james.kraus/.opam/system/lib/base/caml /Users/james.kraus/.opam/system/lib/base /Users/james.kraus/.opam/4.07.0/lib/ocaml
Sending response {"id": 7, "jsonrpc": "2.0", "error": {"code": -32603, "message": "Could not run compiler (ran /Users/james.kraus/.opam/4.07.0/bin/ocamlopt.opt -version). Output: "}}

I'm not sure why it's trying to access /Users/james.kraus/.opam/4.07.0/bin/ocamlopt.opt since /Users/james.kraus/.opam/4.07.0 doesn't exist.

Anyone have any ideas?

Versions of stuff being used:

$ opam --version
2.0.3
$ ocaml --version
The OCaml toplevel, version 4.07.0

I'm using version 1.5.2 of RLS

Edit:

I'm guessing the directory is coming from these lines:

reason-language-server/src/analyze/BuildSystem.re

Lines 209 to 214 in af83153

	/* in local switches that share the sys-ocaml-version, the ocaml library and
	* binaries are apparently in the global (system) switch ¯\_(ツ)_/¯.
	*/
	let%try sysOCamlVersion = getLine("opam config var sys-ocaml-version", ~pwd=root);
	let%try opamRoot = getLine("opam config var root", ~pwd=root);
	Ok(opamRoot /+ sysOCamlVersion /+ path);

Further debug info:

$ opam config env
OPAM_SWITCH_PREFIX='/Users/james.kraus/.opam/system'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='/Users/james.kraus/.opam/system/lib/stublibs:/usr/local/lib/ocaml/stublibs:/usr/local/lib/ocaml'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='/Users/james.kraus/.opam/system/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
PATH='...blah blah blah my path...'; export PATH;

$ opam config list

<><> Global opam variables ><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
arch              x86_64                   # Inferred from system
jobs              4                        # The number of parallel jobs set up in opam configuration
make              make                     # The 'make' command to use
opam-version      2.0.3                    # The currently running opam version
os                macos                    # Inferred from system
os-distribution   homebrew                 # Inferred from system
os-family         homebrew                 # Inferred from system
os-version        10.14.2                  # Inferred from system
root              /Users/james.kraus/.opam # The current opam root directory
switch            system                   # The identifier of the current switch
sys-ocaml-version 4.07.0                   # OCaml version present on your system independently of opam, if any

<><> Configuration variables from the current switch ><><><><><><><><><><><>  🐫
prefix   /Users/james.kraus/.opam/system
lib      /Users/james.kraus/.opam/system/lib
bin      /Users/james.kraus/.opam/system/bin
sbin     /Users/james.kraus/.opam/system/sbin
share    /Users/james.kraus/.opam/system/share
doc      /Users/james.kraus/.opam/system/doc
etc      /Users/james.kraus/.opam/system/etc
man      /Users/james.kraus/.opam/system/man
toplevel /Users/james.kraus/.opam/system/lib/toplevel
stublibs /Users/james.kraus/.opam/system/lib/stublibs
group    group
user     james.kraus

[wiwiwa]

exactly the same problem

[jaredly]

Are you using a global switch? Or a local one?

[jameskraus]

Global for me

[jaredly]

hmmmmm. where does that switch live?

[wiwiwa]

Also global switch, no local switch configured.

I don't know where global switch lives. Is ~/.opam/default a right place?

[jameskraus]

I believe it's in ~/.opam/system

[frejsoya]

I'm not aware of the preferred tools in ocaml (i.e. opam/opamfind etc) to find the compiler. However, I see the same error on OS X with homebrew and a corresponding opam 'system compiler'.

with homebrew ocamlopt.opt is in Cellar/ocaml/4.07.1/bin/ocamlopt.opt (similar for ocamlc)

Instead simply using (with opam)

opam config exec -- ocamlc --version

Alternatively (with findlib)

ocamlfind ocamlc --version

Works just fine without second guessing any paths.

[zli]

Same here, archlinux with the ocaml compiler installed by the distribution's pack manager (instead of opam), using opam v2.0.4 with a single global switch. Guess this is typical for those who just prefer a single compiler backend provided by the OS instead of multiple self-compiled ones.

opam recognizes the existing compiler under the special switch name system and identify its release correctly (e.g. 4.07.1).

$ opam switch
#  switch  compiler             description
→  system  ocaml-system.4.07.1  base-bigarray.~unknown base-threads.base base-unix.base ocaml-system.4.07.0

But unlike other switches fully compiled/placed by opam itself, the path to this switch backend is $OPAMROOT/system rather than $OPAMROOT/4.07.1. Also, you won't find the actual compiler binaries (e.g. ocamlc/ocamlopt) under it (i.e. in $OPAMROOT/system/bin), as they are provided at system level (/usr/lib/ocamlc etc.) and already set in path. Same for the system provided libs (e.g. base-threads, base-unix) which could be absent from $OPAMROOT/system/lib (or just empty directories as placeholders), unlike in other switch backends.

So the path composing algorithm ($OPAMROOT + $sys-ocaml-version + bin|lib) probably won't work for the system switch. The binaries/libraries live in two places in this case, i.e. /usr/bin and /usr/lib/ocaml for the system provided ones, $OPAMROOT/system/bin and $OPAMROOT/system/lib for the rests self-compiled by opam.

The other extension (vscode-reasonml) had no such problem, which seems to rely on opam/ocaml/env commands invocations (or explicit path setting) to get the relevant info instead of computing the paths itself.

The issue #280 looks like a dup of this one. Same as the reporter on that issue, I also make use of VSCode's remote development (SSH) extension (so this extension was automatically installed on the server side), but I guess that is irrelevant.

[jaredly]

@zli If you can determine an algorithm that will reliably find ocamlopt in this situation, I'd love to hear it! The relevant code to modify is here: https://github.com/jaredly/reason-language-server/blob/master/src/analyze/BuildSystem.re#L210

[zli]

@jaredly IIUIC, the system switch is a hard coded special case in opam, so the logic will have to detect this and deal with it separately.

The easiest ways to detect this might be just checking if the current switch is system, alternatively we can check the existence of a package named ocaml-system (a dummy package automatically generated by opam providing the core compiler).

Once we detect the switch is system, then the only reliable way to get the path of ocamlopt seems to ask which to find it. This is because different OSes/distributions can install OCaml under different paths. Since the switch is set to system, it basically means the system provides the ocaml compiler in $PATH, wherever it is.

Note that $OPAMROOT/system/bin and $OPAMROOT/system/lib take precedence over the system default paths (e.g. /usr/bin or /usr/lib where my ocaml compiler and stdlib live). However, as the system compiler binaries and stdlibs are not provided by opam so they do not exist under $OPAMROOT/system, so we'll fallback to the system ones.

The function getOpamLibOrBinPath probably needs refactoring, as the third parameter doesn't make much sense in the case of system switch. Unfortunately I'm still not familiar with the setup of this project, nor should I claim I'm familiar with opam (these are just observations after a quick skim of the opam code), otherwise I'd be happy to contribute patches to fix this.

FYI, here are the output of some opam environment vairalbes on my machine ($OPAMROOT=~/dev/opam)：

$ opam switch
#  switch  compiler             description
→  system  ocaml-system.4.07.1  base-bigarray.~unknown base-threads.base base-unix.base ocaml-system.4.07.0


$ opam config env
OPAM_SWITCH_PREFIX='/home/zheng/dev/opam/system'; export OPAM_SWITCH_PREFIX;
CAML_LD_LIBRARY_PATH='/home/zheng/dev/opam/system/lib/stublibs:/usr/lib/ocaml/stublibs:/usr/lib/ocaml'; export CAML_LD_LIBRARY_PATH;
OCAML_TOPLEVEL_PATH='/home/zheng/dev/opam/system/lib/toplevel'; export OCAML_TOPLEVEL_PATH;
MANPATH=':/home/zheng/dev/opam/system/man'; export MANPATH;
PATH='/home/zheng/dev/opam/system/bin:/home/zheng/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl'; export PATH;


$ opam config list

<><> Global opam variables ><><><><><><><><><><><><><><><><><><><><><><><><><><>
arch              x86_64               # Inferred from system
jobs              1                    # The number of parallel jobs set up in opam configuration
make              make                 # The 'make' command to use
opam-version      2.0.5                # The currently running opam version
os                linux                # Inferred from system
os-distribution   arch                 # Inferred from system
os-family         arch                 # Inferred from system
os-version        unknown              # Inferred from system
root              /home/zheng/dev/opam # The current opam root directory
switch            system               # The identifier of the current switch
sys-ocaml-version 4.07.1               # OCaml version present on your system independently of opam, if any

<><> Configuration variables from the current switch ><><><><><><><><><><><><><>
prefix   /home/zheng/dev/opam/system
lib      /home/zheng/dev/opam/system/lib
bin      /home/zheng/dev/opam/system/bin
sbin     /home/zheng/dev/opam/system/sbin
share    /home/zheng/dev/opam/system/share
doc      /home/zheng/dev/opam/system/doc
etc      /home/zheng/dev/opam/system/etc
man      /home/zheng/dev/opam/system/man
toplevel /home/zheng/dev/opam/system/lib/toplevel
stublibs /home/zheng/dev/opam/system/lib/stublibs
group    users
user     zheng

$ which ocamlopt
/usr/bin/ocamlopt

$ ocamlopt -where
/usr/lib/ocaml

$ ls /usr/bin/ocaml*
/usr/bin/ocaml        /usr/bin/ocamlc.opt    /usr/bin/ocamldebug     /usr/bin/ocamldoc       /usr/bin/ocaml-instr-graph   /usr/bin/ocamllex.opt     /usr/bin/ocamlmktop       /usr/bin/ocamlobjinfo.byte  /usr/bin/ocamlopt.opt    /usr/bin/ocamlprof       /usr/bin/ocamlrund
/usr/bin/ocamlc       /usr/bin/ocamlcp       /usr/bin/ocamldep       /usr/bin/ocamldoc.opt   /usr/bin/ocaml-instr-report  /usr/bin/ocamlmklib       /usr/bin/ocamlmktop.byte  /usr/bin/ocamlobjinfo.opt   /usr/bin/ocamloptp       /usr/bin/ocamlprof.byte  /usr/bin/ocamlruni
/usr/bin/ocamlc.byte  /usr/bin/ocamlcp.byte  /usr/bin/ocamldep.byte  /usr/bin/ocamlfind      /usr/bin/ocamllex            /usr/bin/ocamlmklib.byte  /usr/bin/ocamlmktop.opt   /usr/bin/ocamlopt           /usr/bin/ocamloptp.byte  /usr/bin/ocamlprof.opt   /usr/bin/ocaml-syntax-shims
/usr/bin/ocamlcmt     /usr/bin/ocamlcp.opt   /usr/bin/ocamldep.opt   /usr/bin/ocamlfind_opt  /usr/bin/ocamllex.byte       /usr/bin/ocamlmklib.opt   /usr/bin/ocamlobjinfo     /usr/bin/ocamlopt.byte      /usr/bin/ocamloptp.opt   /usr/bin/ocamlrun        /usr/bin/ocamlyacc

$ ls ~/dev/opam/system/bin/
bdump  cppo  dune  jbuilder  ocaml  ocamlbuild  ocamlbuild.byte  ocamlbuild.native  ocamlfind  ocamlmerlin  ocamlmerlin-server  ocaml-syntax-shims  ocp-autoconf  ocp-autoconf.byte  ocp-build.byte  ocp-grep  ocp-indent  ocp-index  ocp-pp  ocp-pp.byte  ppx_lwt  safe_camlp4  ydump