Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for PyTables's filenodes #81

Closed
mivade opened this issue Sep 6, 2017 · 2 comments
Closed

Add support for PyTables's filenodes #81

mivade opened this issue Sep 6, 2017 · 2 comments

Comments

@mivade
Copy link
Contributor

mivade commented Sep 6, 2017

Currently PyTables filenodes are displayed as an array of integers. It would be nice if ViTables detected that it was a filenode and display as a (byte) string instead. This is detectable via the NODE_TYPE attribute with value file.

@uvemas
Copy link
Owner

uvemas commented Sep 21, 2017

Done in a0852cc. The support is basic, not very sophisticated. A filenode with just 100000 short lines takes about 40 seconds to open on my laptop because a VLArray container needs to be created and populated. Once this is done you can close and open the filenode as fast as any other dataset. I will try to optimize the current implementation but right now I think it is not easy (but maybe I'm a little bit dense right now :-). Suggestions are welcomed.

@uvemas
Copy link
Owner

uvemas commented Sep 23, 2017

Things work better in d45f008. Now a filenode with one million short lines takes about 40 seconds to open for the very first time so opening time has improved one order of magnitude. It is still too poor performance but I hope it helps you. Unfortunately I don't know how to keep improving the performance so I'm closing the bug. Please, let me know if you have some suggestion.

@uvemas uvemas closed this as completed Sep 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants