Skip to content

Commit

Permalink
Modularize (#8)
Browse files Browse the repository at this point in the history
  • Loading branch information
@qubixes
qubixes authored Mar 26, 2020
1 parent 10ee615 commit f52eb43
Show file tree
Hide file tree
Showing 13 changed files with 561 additions and 243 deletions.
46 changes: 46 additions & 0 deletions .github/workflows/ci-workflow.yml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: test-suite
on: [push, pull_request]
jobs:
test-master:
name: pytest
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
path: asr-plot

- uses: actions/checkout@v2
with:
repository: asreview/asreview
path: asr-core
- uses: actions/setup-python@v1
with:
python-version: '3.6'
architecture: 'x64'
- name: Install packages and run tests
run: |
pip install pytest
pip install --upgrade setuptools>=41.0.0
pip install ./asr-core[all]
pip install ./asr-plot
pytest asr-plot/tests
#test-older:
#name: pytest
#runs-on: ubuntu-latest
#strategy:
#matrix:
#asr_versions: ['0.6', '0.7', '0.7.1', '0.7.2']
#steps:
#- uses: actions/checkout@v2
#- uses: actions/setup-python@v1
#with:
#python-version: '3.6'
#architecture: 'x64'
#- name: Install packages and run tests
#run: |
#pip install pytest
#pip install --upgrade setuptools>=41.0.0
#pip install asreview[all]==${{ matrix.asr_versions }}
#pip install .
#pytest tests
62 changes: 48 additions & 14 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
## ASReview-visualization
# ASReview-visualization

![Deploy and release](https://github.com/msdslab/asreview-visualization/workflows/Deploy%20and%20release/badge.svg)
![Deploy and release](https://github.com/asreview/asreview-visualization/workflows/Deploy%20and%20release/badge.svg)![Build status](https://github.com/asreview/asreview-visualization/workflows/test-suite/badge.svg)

This is a plotting/visualization supplemental package for the
[ASReview](https://github.com/asreview/asreview)
software. It is a fast way to create a visual impression of the ASReview with different
dataset, models and model parameters.

### Installation
## Installation

The easiest way to install the visualization package is to use the command line:

Expand All @@ -24,11 +24,11 @@ asreview --help

It should list the 'plot' modus.

### Basic usage
## Basic usage

Log files that were created with the same ASReview settings can be put together/averaged by putting
them in the same directory. Log files with different settings/datasets should be put in different
directories to compare them. It is advised to put these log files in the same directory.
State files that were created with the same ASReview settings can be put together/averaged by putting
them in the same directory. State files with different settings/datasets should be put in different
directories to compare them.

As an example consider the following directory structure, where we have two datasets, called `ace` and
`ptsd`, each of which have 8 runs:
Expand Down Expand Up @@ -68,17 +68,18 @@ asreview plot ace ptsd --absolute-values
```


### Plot types
## Plot types

There are currently three plot types implemented: _inclusions_, _discovery_, _limits_. They can be
individually selected with the `-t` or `--type` switch. Multiple plots can be made by using `,` as
a separator:
There are currently four plot types implemented:
_inclusion_, _discovery_, _limit_, _progression_.
They can be individually selected with the `-t` or `--type` switch. Multiple plots
can be made by using `,` as a separator:

```bash
asreview plot ace ptsd --type 'inclusions,discovery'
```

#### Inclusions
### Inclusion

This figure shows the number/percentage of included papers found as a function of the
number/percentage of papers reviewed. Initial included/excluded papers are subtracted so that the line
Expand All @@ -88,7 +89,7 @@ The quicker the line goes to a 100%, the better the performance.

![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/inclusions.png?raw=true "Inclusions")

#### Discovery
### Discovery

This figure shows the distribution of the number of papers that have to be read before discovering
each inclusion. Not every paper is equally hard to find.
Expand All @@ -98,7 +99,7 @@ The closer to the left, the better.
![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/discovery.png?raw=true "Discovery")


#### Limits
### Limit

This figure shows how many papers need to be read with a given criterion. A criterion is expressed
as "after reading _y_ % of the papers, at most an average of _z_ included papers have been not been
Expand All @@ -109,3 +110,36 @@ _z_ are 0.1, 0.5 and 2.0.
The quicker the lines touch the black (`y=x`) line, the better.

![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/limits.png?raw=true "Limits")

### Progression

This figure shows the average inclusion rate as a function of time, number of papers read.
The more concentrated on the left, the better. The thick line is the average of individual runs
(thin lines). The visualization package will automatically detect which are directories and which
are files. The curve is smoothed out by using a Gaussian smoothing algorithm.

![alt text](https://github.com/msdslab/asreview-visualization/blob/master/docs/progression.png?raw=true "Progression")


## API

To make use of the more advanced features, you can also use the visualization package
as a library. The advantage is that you can make more reproducible plots where text, etc. is
in the place *you* want it. Examples can be found in module `asreviewcontrib.visualization.quick`.
Those are the scripts that are used for the command line interface.

```python
with Plot.from_paths(["PATH_1", "PATH_2"]) as plot:
inc_plot = plot.new("inclusion")
inc_plot.set_grid()
inc_plot.set_xlim(0, 30)
inc_plot.set_ylim(0, 101)
inc_plot.set_legend()
inc_plot.show()
inc_plot.save("SOME_FILE.png")
```

Of course fill in `PATH_1` and `PATH_2` as the files you would like to plot.

If the customization is not sufficient, you can also directly manipulate the `self.ax` and
`self.fig` attributes of the plotting class.
48 changes: 22 additions & 26 deletions asreviewcontrib/visualization/entrypoint.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,18 @@
# limitations under the License.

import argparse
import logging

from asreview.config import LOGGER_EXTENSIONS
from asreview.entry_points import BaseEntryPoint

from asreviewcontrib.visualization import Plot
import logging
from asreviewcontrib.visualization.quick import progression_plot
from asreviewcontrib.visualization.quick import limit_plot
from asreviewcontrib.visualization.quick import discovery_plot
from asreviewcontrib.visualization.quick import inclusion_plot

PLOT_TYPES = ['inclusions', 'discovery', 'limits']
PLOT_TYPES = ['inclusion', 'discovery', 'limit', 'progression']


class PlotEntryPoint(BaseEntryPoint):
Expand Down Expand Up @@ -54,23 +58,25 @@ def execute(self, argv):
else:
result_format = "percentage"

output = args_dict["output"]

prefix = args_dict["prefix"]
legend = not args_dict["no_legend"]
with Plot.from_paths(args_dict["data_paths"], prefix=prefix) as plot:
if len(plot.analyses) == 0:
print(f"No log files found in {args_dict['data_paths']}.\n"
f"To be detected log files have to start with '{prefix}'"
f" and end with one of the following: \n"
f"{', '.join(LOGGER_EXTENSIONS)}.")
return
if "inclusions" in types:
plot.plot_inc_found(result_format=result_format, legend=legend,
abstract_only=args_dict["abstract_only"],
wss_value=args_dict["wss_value"])

if "inclusion" in types:
inclusion_plot(plot, output=output, result_format=result_format) # noqa
if "discovery" in types:
plot.plot_time_to_discovery(result_format=result_format)
if "limits" in types:
plot.plot_limits(result_format=result_format)
discovery_plot(plot, output=output, result_format=result_format) # noqa
if "limit" in types:
limit_plot(plot, output=output, result_format=result_format) # noqa
if "progression" in types:
progression_plot(plot, output=output, result_format=result_format) # noqa


def _parse_arguments():
Expand Down Expand Up @@ -102,21 +108,11 @@ def _parse_arguments():
'starting with a prefix.'
)
parser.add_argument(
"--abstract_only",
default=False,
action="store_true",
help="Use after abstract screening as the inclusions/exclusions."
)
parser.add_argument(
"--no_legend",
default=False,
action="store_true",
help="Don't show a legend with the plot."
)
parser.add_argument(
"--wss_value",
default=False,
action="store_true",
help="Add WSS values to plot."
"-o", "--output",
default=None,
help='Save the plot to a file. If multiple plots are made, only one'
' is saved (non-deterministically). File formats are detected '
' by the matplotlib library, check there to see available '
'formats.'
)
return parser
Loading

0 comments on commit f52eb43

Please sign in to comment.