Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Marques score assertion errors #1197

Open
samwinebrake opened this issue Aug 22, 2024 · 0 comments
Open

Marques score assertion errors #1197

samwinebrake opened this issue Aug 22, 2024 · 0 comments
Assignees
Labels

Comments

@samwinebrake
Copy link
Collaborator

Score assertion errors are present within the marques2020 benchmark tests. See below for an example:

brainscore_vision/benchmarks/marques2020/test.py:68 (test_Marques2020[Marques2020_Cavanaugh2002-grating_summation_field-expected0])
<xarray.Score ()>
array(0.77613597)
Attributes:
    error:    0.024806444383184106
    raw:      [0.79613484 0.76925312 0.7953668  0.77673325 0.79324324 0.76894... != 0.599 ± 1.0e-02

<Click to see difference>

benchmark = 'Marques2020_Cavanaugh2002-grating_summation_field'
expected = 0.599 ± 1.0e-02

Also, to get to the point of having this test run, --target8.00--sourceNone must be stripped from the stimuli.identifer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants