errors trying to use the sampler for energy-based sampling

[alecristia]

recordings option

I see here that an edit allowing me to pass a recordings parameter to choose the recordings have been merged. However, even after updating childproject, I still get an error:

$ child-project sampler . och_mal/ energy-detection --windows-length 30000 --windows-spacing 300000 --windows-offset 1800000 --windows-count 40 --threshold .75 --low-freq 50 --high-freq 3000 --by recording_filename --recordings samples/selrecs.csv

usage: child-project [-h]
{validate,import-annotations,merge-annotations,intersect-annotations,remove-annotations,rename-annotations,import-data,overview,compute-durations,process,sampler,zooniverse,eaf-builder,anonymize}
...
child-project: error: unrecognized arguments: --recordings samples/selrecs.csv

To update the ChildProject package, I did pip3 install git+https://github.com/LAAC-LSCP/ChildProject.git.

I do see that the option is not yet in the docs, so I may be misunderstanding its availability!

recordings not found

$ child-project sampler . och_mal/ energy-detection --windows-length 30000 --windows-spacing 300000 --windows-offset 1800000 --windows-count 40 --threshold .75 --low-freq 50 --high-freq 3000
yields

failed to read './recordings/raw/COF_030503_1', is it a valid audio file ?

I figured this may be due to the fact that we haven't yet split up audio files, so I renamed the file _1 by hand, but it's still not found. Is it silly to think that this may have to do with the absence of extension in the recording filename (in the metadata)?

e.g.:

$ ls ./recordings/raw/COF_030503_1*
./recordings/raw/COF_030503_1.wav

(the file itself is definitely there)

[lucasgautheron]

recordings option

I see here that an edit allowing me to pass a recordings parameter to choose the recordings have been merged. However, even after updating childproject, I still get an error:

$ child-project sampler . och_mal/ energy-detection --windows-length 30000 --windows-spacing 300000 --windows-offset 1800000 --windows-count 40 --threshold .75 --low-freq 50 --high-freq 3000 --by recording_filename --recordings samples/selrecs.csv

usage: child-project [-h]
{validate,import-annotations,merge-annotations,intersect-annotations,remove-annotations,rename-annotations,import-data,overview,compute-durations,process,sampler,zooniverse,eaf-builder,anonymize}
...
child-project: error: unrecognized arguments: --recordings samples/selrecs.csv

To update the ChildProject package, I did pip3 install git+https://github.com/LAAC-LSCP/ChildProject.git.

I do see that the option is not yet in the docs, so I may be misunderstanding its availability!

Actually it is in the docs, but it may be tricky: https://childproject.readthedocs.io/en/latest/samplers.html#overview

The --recordings should be passed before tenergy-detection; this is true for all the switches that are common to all samplers.
So it should be:

child-project sampler . och_mal/ --recordings samples/selrecs.csv \
 energy-detection \
 --windows-length 30000 \
 --windows-spacing 300000 \
 --windows-offset 1800000 \
 --windows-count 40 \
 --threshold .75 \
 --low-freq 50 \
 --high-freq 3000 \
 --by recording_filename

recordings not found

$ child-project sampler . och_mal/ energy-detection --windows-length 30000 --windows-spacing 300000 --windows-offset 1800000 --windows-count 40 --threshold .75 --low-freq 50 --high-freq 3000
yields

failed to read './recordings/raw/COF_030503_1', is it a valid audio file ?

I figured this may be due to the fact that we haven't yet split up audio files, so I renamed the file _1 by hand, but it's still not found. Is it silly to think that this may have to do with the absence of extension in the recording filename (in the metadata)?

e.g.:

$ ls ./recordings/raw/COF_030503_1*
./recordings/raw/COF_030503_1.wav

(the file itself is definitely there)

You're right, recording_filename should include the .wav extension.

[alecristia]

oops, sorry I missed that!

About the second error, just to keep going, I removed the wav extension of the sound file, and now I get a new error, which I'm not sure how to interpret:

(ChildProjectVenv) (base) Mac-mini:lyon acristia$ child-project sampler . samples/och_mal/ --recordings samples/selrecs.csv energy-detection --windows-length 30000 --windows-spacing 300000 --windows-offset 1800000 --windows-count 40 --threshold .75 --low-freq 50 --high-freq 3000

Traceback (most recent call last):
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2895, in get_loc
return self._engine.get_loc(casted_key)
File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
File "pandas/_libs/hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas/_libs/hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'recording_filename'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/acristia/ChildProjectVenv/bin/child-project", line 11, in
load_entry_point('ChildProject==0.0.1', 'console_scripts', 'child-project')()
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/ChildProject/cmdline.py", line 331, in main
args.func(args)
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/ChildProject/cmdline.py", line 31, in
_parser.set_defaults(func = lambda args: cls().run(**vars(args)))
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/ChildProject/pipelines/samplers.py", line 574, in run
splr = EnergyDetectionSampler(self.project, **kwargs)
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/ChildProject/pipelines/samplers.py", line 309, in init
super().init(project, recordings)
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/ChildProject/pipelines/samplers.py", line 43, in init
self.recordings = pd.read_csv(recordings)['recording_filename'].tolist()
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/pandas/core/frame.py", line 2906, in getitem
indexer = self.columns.get_loc(key)
File "/Users/acristia/ChildProjectVenv/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc
raise KeyError(key) from err
KeyError: 'recording_filename'

[lucasgautheron]

It looks like samples/selrecs.csv is missing a 'recording_filename' column ?

[alecristia]

ah, of course! sorry

[lucasgautheron]

Just made the error message more explicit 85ec0a7