nickel startup overhead seems significant.

[suimong]

This is not a "request" in any way, just a nice-to-have sort of issue.

I noticed that nickel's startup time is "slow": on my machine it took about 35ms to execute a minimal '{a = 1} | nickel eval'. For reference, a minimal python invocation python -c '1 + 1' took 15ms.

In most circumstances this doesn't matter. But I noticed the lag because I was using nickel in a nushell module to validate a bunch of json files during import. Nickel binary is called 8 times, which costs ~400ms, leading to the noticeable lag for the nushell module import.

I wonder if there are low hanging fruits that could improve the startup time.

[jneem]

I ran into something similar when benchmarking package manifest evaluation. It turns out that there's a substantial (~20ms on my machine) amount of time spent parsing the standard library. We embed the standard library source in the nickel binary, but parse it on every evaluation.

Possible mitigations:

• store some intermediate representation that's faster to load
• allow multiple evaluations on a single run. You'd still pay the 35ms, but not the 400ms from running 8 times.

[suimong]

It turns out that there's a substantial (~20ms on my machine) amount of time spent parsing the standard library.

That explains. Thanks!

store some intermediate representation that's faster to load

I imagine if the parsed stdlib is somehow serializable (in a straightforward way a la serde) then this approach would be quite feasible? Otherwise, it sounds like lots of work for relatively little gain.

allow multiple evaluations on a single run. You'd still pay the 35ms, but not the 400ms from running 8 times.

This is an interesting approach, and paying the startup overhead once is definitely acceptable, since that's the minimum cost no matter what. Though the CLI interface might be the tricky part. On top of my head I can't think of a CLI program that enables such "multiple input, multiple output" paradigm. Also in my case, there are calls to both nickel eval and nickel export among the 8 times nickel gets called, so if we were to allow multiple evaluations per single run then it should also take different subcommands into consideration.

"lazy" parsing of stdlib is another approach, just to add to the ideas list.

[yannham]

Given RFC007 is in the process of being implemented, I would be inclined to wait to see the result first. In particular once we have a bytecode VM, there's a natural, serializable format to store the stdlib already processed: bytecode.

It might take a bit of time to land still, so if it's really an important problem, maybe one light work-around could be considered in the meantime.

Otherwise, there's a --no-stdlib flag for tests which is currently only available in debug mode. However it would be a one-line patch to make it always available. If you know your file won't use the stdlib because it's very simple, then it might provide an improved start up time.

[suimong]

Thanks for mentioning the --no-stdlib flag, did not know about that before. It's not urgent at all, and I can work around that by executing nickel calls in parallel.

Looking forward to seeing the bytecode VM landed!