Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getter Docstrings #1185

Merged
merged 21 commits into from
Feb 7, 2025
Merged

Getter Docstrings #1185

merged 21 commits into from
Feb 7, 2025

Conversation

pauladkisson
Copy link
Member

@pauladkisson pauladkisson commented Jan 24, 2025

Added Returns section to all getter docstrings.

Fixes #1097

Cline (Claude 3.5 Sonnet) helped. It was so close to just getting it right after the first couple attempts, but then took a lot of coaching to get those last few...

@pauladkisson
Copy link
Member Author

pauladkisson commented Jan 28, 2025

After the meeting today, I think this kind of task would be a really good benchmark for AI agents / LLMs / prompts.

Some quantifiable questions:

  • how many true positive? (docstrings that needed to be edited that were CORRECTLY edited)
  • how many false positives? (docstrings that DIDNT need to be edited that were INCORRECTLY edited)
  • how many false negatives? (docstrings that needed to be edited that were MISSED or edited INCORRECTLY)
  • how many attempts before it completes the task? (In this case I had to re-prompt cline with different instructions 7 times until it finally got all of the docstrings I wanted)
  • Total cost
  • Total runtime

With that in mind, it might be worthwhile to branch off from neuroconv with a dedicated ai_docstrings_benchmark branch before merging these changes.


Returns
-------
type
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not useful, what do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is actually fairly reasonable. The docstring doesn't provide a ton of info, but it stays consistent with the standard of the repo.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant the type no the whole docstring.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's accurate right? type(class) is type.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accurate I agree but I don't think it adds too much. There is an Any somewhere there as well.

Just commenting, don't really know how I feel, I don't think it subtracts.

Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an improvement to me, some small comments but should be good to go after that.

src/neuroconv/tools/roiextractors/roiextractors.py Outdated Show resolved Hide resolved
@@ -99,7 +113,14 @@ def get_metadata_schema(self) -> dict:
return metadata_schema

def get_metadata(self) -> DeepDict:
"""Auto-fill as much of the metadata as possible. Must comply with metadata schema."""
"""
Auto-fill as much of the metadata as possible. Must comply with metadata schema.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not useful this one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me.

src/neuroconv/datainterfaces/behavior/video/video_utils.py Outdated Show resolved Hide resolved
src/neuroconv/datainterfaces/behavior/video/video_utils.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@h-mayorquin h-mayorquin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pauladkisson pauladkisson merged commit 6744dab into main Feb 7, 2025
40 checks passed
@pauladkisson pauladkisson deleted the returns branch February 7, 2025 18:45
Copy link

codecov bot commented Feb 7, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.65%. Comparing base (96dfdff) to head (2ce0742).
Report is 31 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1185      +/-   ##
==========================================
- Coverage   90.69%   89.65%   -1.05%     
==========================================
  Files         129      129              
  Lines        8189     8378     +189     
==========================================
+ Hits         7427     7511      +84     
- Misses        762      867     +105     
Flag Coverage Δ
unittests 89.65% <100.00%> (-1.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/neuroconv/basedatainterface.py 80.21% <ø> (-15.13%) ⬇️
src/neuroconv/baseextractorinterface.py 100.00% <ø> (ø)
...oconv/datainterfaces/behavior/video/video_utils.py 97.14% <100.00%> (ø)
...atainterfaces/behavior/video/videodatainterface.py 93.54% <ø> (ø)
...roconv/datainterfaces/ecephys/axona/axona_utils.py 57.94% <ø> (ø)
...erfaces/ecephys/baserecordingextractorinterface.py 93.87% <ø> (+0.06%) ⬆️
...nterfaces/ecephys/basesortingextractorinterface.py 80.19% <ø> (ø)
...ainterfaces/ecephys/neuroscope/neuroscope_utils.py 100.00% <ø> (ø)
...aces/ecephys/neuroscope/neuroscopedatainterface.py 93.65% <ø> (-2.39%) ⬇️
.../ecephys/openephys/openephysbinarydatainterface.py 97.29% <ø> (ø)
... and 27 more

... and 9 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Documentation]: Should we add Returns section to the docstrings of our get methods (get_metadata, etc.)
2 participants