Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make loading features more efficient #48

Open
SkepticRaven opened this issue Nov 18, 2024 · 0 comments
Open

Make loading features more efficient #48

SkepticRaven opened this issue Nov 18, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@SkepticRaven
Copy link
Contributor

Right now loading of features has a lot of room for performance improvements. Larger projects can take more time reading in features than training the classifiers takes, particularly if long videos are used with very sparse labels.

Brief rundown of current strategy:

  1. Load labels from project
  2. Initialize feature object, which loads all features in the file
  3. Load selected features from feature object here
  4. Feature loading accesses features through feature_module.items().
  5. Subset features to only labeled frames (unnecessary? should already be subset above)

Proposed change:
Option 1:
Change initialization strategy to delay reading in or calculating features until its needed. There's a lot of stuff that accesses the feature functions and we would need to check testing coverage.
Option 2:
Make a static function for IdentityFeatures that will read in selectively. We would need to be careful about handling the situation where the data may not be cached, which is a bit more difficult for window features.

@SkepticRaven SkepticRaven added the enhancement New feature or request label Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant