Fix IBL streaming tests

[alejoe91]

Somehow the same PID resulted in a different number of units.

@oliche anything changed on the ONE-api side? Is it possible that a PID is now pointing to a different insertion?

[zm711]

This seems really risky to have a gt data set and then just test that we load that dataset. Especially if they can change. When I was looking at the failed tests it also said a failure between subtracting datetime.datetime and NoneType. Is that error related to this too? Seems like changes in IBL overall no?

[alejoe91]

Yeah maybe we can pin it to a previous version? I agree that probably we don't need to assert the number of units. Maybe just asserting that good units are less than all?

[zm711]

That seems safer for a test from my perspective.

[oliche]

Yes, I can show you how to use the revision argument that pins the query to a version of the dataset.

[alejoe91]

Thanks that would be great!

[oliche]

Hi Alessio,
This is what I would propose to pin the IBL dataset and also provide the users with a way to do it themselves if they wish so. alejoe91#25

[alejoe91]

@zm711 I think that after adding revision we should keep the current behavior to test that the IBL retriever is actually getting the right data!

Reayd to merge on my end

[zm711]

@zm711 I think that after adding revision we should keep the current behavior to test that the IBL retriever is actually getting the right data!

Reayd to merge on my end

Sounds good to me!

[zm711]

And this should fix the datetime issue too?

[alejoe91]

And this should fix the datetime issue too?

No :( @oliche can you help us debug this error as well?

self.ssl = SpikeSortingLoader(one=one, pid=pid)
<string>:18: in __init__
    ???
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/brainbox/io/one.py:815: in __post_init__
    self.eid, self.pname = self.one.pid2eid(self.pid)
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/one/util.py:162: in wrapper
    self.refresh_cache(mode=mode)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = One (online, https://openalyx.internationalbrainlab.org),/ mode = 'auto'

    def refresh_cache(self, mode='auto'):
        """Check and reload cache tables.
    
        Parameters
        ----------
        mode : {'local', 'refresh', 'auto', 'remote'}
            Options are 'local' (don't reload); 'refresh' (reload); 'auto' (reload if expired);
            'remote' (don't reload).
    
        Returns
        -------
        datetime.datetime
            Loaded timestamp.
        """
        # NB: Currently modified table will be lost if called with 'refresh';
        # May be instances where modified cache is saved then immediately replaced with a new
        # remote cache. Also it's too slow :(
        # self.save_cache()  # Save cache if modified
        if mode in {'local', 'remote'}:
            pass
        elif mode == 'auto':
>           if datetime.now() - self._cache['_meta']['loaded_time'] >= self.cache_expiry:
E           TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'