Skip to content

Commit

Permalink
Update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
caufieldjh committed Oct 20, 2023
1 parent 019e6f8 commit 5a600dd
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 1 deletion.
6 changes: 5 additions & 1 deletion docs/evaluations.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Tests may be run with the `eval` command followed by the test name, e.g., for th
ontogpt eval EvalCTD
```

By default, the evaluation will only be executed over a subset of the test input.
By default, the evaluation will only be executed over a subset of the test corpus.

The exact number of inputs to run the test over can be controlled with the `--num-tests` option, like this:

Expand All @@ -20,6 +20,10 @@ ontogpt eval --num-tests 1 EvalCTD

To run the full set of tests, set `num-tests` to the input count for a given evaluation, as defined below.

For each document, the evaluation process will attempt to process the full text without preprocessing.

If the `--chunking` option is used, then the input text will instead be chunked into segments of a few tokens each, essentially creating new queries for each segment. This alternate strategy may impact test results and will result in longer run time.

## BC5CDR

*Test Name:* EvalCTD
Expand Down
1 change: 1 addition & 0 deletions docs/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,7 @@ See the Evaluations section for more details.
Options:

* `--num-tests INTEGER` - number of test iterations to cycle through. Defaults to 5.
* `--chunking` / `--no-chunking` - If set, chunk input text, then prepare a separate prompt for each chunk. Otherwise the full input text is passed. Defaults to False.

Example:

Expand Down

0 comments on commit 5a600dd

Please sign in to comment.