Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

long_field_names option when writing neo blocks in .mat files #1636

Open
TommasoLambresa opened this issue Jan 28, 2025 · 1 comment
Open
Labels

Comments

@TommasoLambresa
Copy link

Describe the bug
We're trying to load an Intan .rhs file using neo.io.IntanIO module and then writing it into a .mat file using neo.io.NeoMatlabIO.
When doing this, the editor raises "ValueError: Field names are restricted to 31 characters" relating to the scipy.io.savemat function. This is because the write method within NeoMatlabIO object doesn't pass any additional argument than the block to the write_block method, even if the function accepts **kargs which remains unused within it.
The issue could be easily solved by adding the argument long_field_names=True as input to scipy.io.savemat function, which expands to 64 the maximum number of characters for a field.

To Reproduce
The code that we are using is:

from neo import Block
from neo.io import IntanIO, NeoMatlabIO

r = IntanIO(filename='my/rhs/file')
w = NeoMatlabIO(filename='convertedfile.mat')
blocks = r.read()
w.write(blocks[0]) 

the analogsig object within the block contains an annotation called desired_impedance_test_frequency which has 32 characters, causing the issue.

Expected behaviour
We propose a minimal modification to the write_block method within the NeoMatlabIO class:

  • adding the **kargs as input to scipy.io.savemat: this won't cause any issue because it remains unused within the code and will allow to pass any argument accepted by the savemat function directly in the write method at the begininng, that in our case is long_field_names=True.

We have tested and with this modification all works as expected.
A brief description of the proposed changes to enhance clarity:

class NeoMatlabIO(BaseIO):

...

    def write_block(self, bl, **kargs):
        """
        Arguments:
            bl: the block to be saved
        """
        import scipy.io

       ...

        scipy.io.savemat(self.filename, {"block": bl_struct}, oned_as="row", **kargs)

this reflected in our code in:

from neo import Block
from neo.io import IntanIO, NeoMatlabIO

r = IntanIO(filename='my/rhs/file')
w = NeoMatlabIO(filename='convertedfile.mat')
blocks = r.read()
w.write(blocks[0], long_field_names = True) 

Environment:

  • OS: Windows 11 Pro
  • Python 3.10.14
  • Neo 0.14.0
  • NumPy 2.2.2
@zm711
Copy link
Contributor

zm711 commented Jan 28, 2025

I would suggest creating the PR here. I think it should be fine, but would love to make sure that this doesn't break our CI. We can come up with a test to add as well in the PR. @apdavison is probably the most expert at this IO. But my gut feeling is submit the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants