Julia engine commands

[jkrumbiegel]

Description

This PR adds CLI commands to interact with or diagnose the native julia engine. It adds a new subcommand engine for this purpose:

$ quarto engine            

Usage:   quarto engine
Version: 99.9.9       

Description:

  Access functionality specific to quarto's different rendering engines.

Options:

  -h, --help  - Show this help.  

Commands:

  help   [command]  - Show this help or the help of a sub-command.                
  julia             - Access functionality specific to the julia rendering engine.

Implementation

The PR adds a new optional field populateCommand to the ExecutionEngine interface which only the julia engine currently sets, but others could potentially as well. This field takes a function that adds its own cliffy subcommands under its key, in the julia engine's case julia.

Julia commands

The subcommands for a given engine can be inspected as well. In this PR I've added status, kill and log.

$ quarto engine julia

Usage:   quarto engine julia
Version: 99.9.9             

Description:

  Access functionality specific to the julia rendering engine.

Options:

  -h, --help              - Show this help.                             
  --log         <file>    - Path to log file                            
  --log-level   <level>   - Log level (info, warning, error, critical)  
  --log-format  <format>  - Log format (plain, json-stream)             
  --quiet                 - Suppress console output.                    
  --profile               - Active project profile(s)                   

Commands:

  help    [command]  - Show this help or the help of a sub-command.                               
  status             - Get status information on the currently running Julia server process       
  kill               - Kills the control server if it is currently running.                       
  log                - Prints the julia server log file if it exists which can be used to diagnose
                       problems. 

The status command can be used to print out information about the currently running server process. For example, if quarto render scratch/test.qmd has just been started the output is:

quarto engine julia status
Julia server is responding.
  port: 8003
  pid: 48443
  julia version: 1.10.5
  environment: /Users/krumbiegel/.julia/dev/QuartoNotebookRunner
  runner version: 0.11.7
  workers active: 1
    worker 1:
      path: /Users/krumbiegel/dev/quarto-cli/scratch/test.qmd      
      run started: 3:48:01 PM (6 seconds ago)    
      run finished: -    
      timeout: 1000 seconds 

or when some notebooks are idling and waiting to be rerun:

$ quarto engine julia status
Julia server is responding.
  port: 8003
  pid: 48443
  julia version: 1.10.5
  environment: /Users/krumbiegel/.julia/dev/QuartoNotebookRunner
  runner version: 0.11.7
  workers active: 1
    worker 1:
      path: /Users/krumbiegel/dev/quarto-cli/scratch/test.qmd      
      run started: 3:48:01 PM (1 minute 2 seconds ago)    
      run finished: 3:48:11 PM (took 10 seconds)    
      timeout: 1000 seconds (15 minutes 48 seconds left)

Before, there was no way to check what had gone wrong if the server didn't respond anymore. Now, the julia process writes to a log file which can be inspected with the log command:

$ quarto engine julia log
[ Info: Log started at 2025-01-06T15:44:19.491
ERROR: LoadError: something failed
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] top-level scope
   @ ~/dev/quarto-cli/src/resources/julia/quartonotebookrunner.jl:92
in expression starting at /Users/krumbiegel/dev/quarto-cli/src/resources/julia/quartonotebookrunner.jl:92

If nothing helps or if the server should be restarted for some other reason, the kill command sends SIGTERM to the pid that's saved in the transport file.

$ quarto engine julia kill
Sent SIGTERM to server process

Trying this PR

This PR depends on unreleased changes to QuartoNotebookRunner.jl. You can clone the repository and check out PR 229. Then you can run quarto with QUARTO_JULIA_PROJECT=path/to/cloned/repo.

Summary

This PR adds julia-engine specific commands to the quarto CLI interface. Having such an engine namespace was discussed earlier with @cscheid but this PR is the first specific proposal how that logic could look. Other than creating the means for other engines to hook in their own commands, this PR does not add any functionality for non-julia engines.

At the current stage I would first like to get maintainer approval for the general mechanism I introduced here. Comments about the interface for the actual commands is appreciated as well. I will probably also need a little help once the PR goes into the test writing stage, as I have only interacted with the notebook rendering test infrastructure so far, not the general tests for the CLI.

[cscheid]

I want to give a little bit of further thought to the engine name for two reasons:

• a top-level command will live as long as quarto itself
• our public top-level commands are all supposed to be verbs

quarto configure engine is a bit a of handful to type, but it reads much better when changing settings. It doesn't make sense for the options here, but I think we need something like that.

[jkrumbiegel]

a top-level command will live as long as quarto itself

I completely understand, which is why I wanted to get your feedback early :)

our public top-level commands are all supposed to be verbs

I see, that didn't occur to me before. Not sure about configure engine given status and log but might make sense in the bigger picture. Maybe it could be allowed to use configure julia directly as a shortcut, then only the internal engine names would have to be avoided if other configurable things are added over time. configure engine xyz could be a fallback for future extensions.

[jkrumbiegel]

@cscheid do you want me to go ahead and switch this PR to configure engine julia?