Create FX Functions #10
Conversation
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]>
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]>
Concerns: `audio` is an int16 array, so this may lead to rounding and quantization errors. Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Focused on `change_speed` and making sure we return copies of mutable objects Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Thanks to @irinazoccolini ! Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
to reflect the extra `score` value included in each tuple in the `notes` list Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
(It doesn't exactly represent the period of the sine wave but the amount of time for which it would play) Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
The `change_speed` function works find for speeding up audio, but we need a solution for slowing down audio (we need more samples than we're given!) Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Note added to reveal the very narrow assumptions on which these functions operate; still working out the `overlap_notes` filter Co-Authored-By: Watermelanie <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]>
All filters (except `overlap_notes`) will act on a note at a time and will be applied via the `apply_filter` function. Order of priorities: `_get_notes`, `_get_frequency`, `change_pitch`, `overlap_notes`, and (if time permits), a second look at `change_volume`. All other functions should have a stable implementation, but we likely cannot test anything until the first three functions listed above are fully implemented. Co-Authored-By: Watermelanie <[email protected]> Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]>
(All located in a new `assets` directory)
Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: Watermelanie <[email protected]>
Added constant dictionary to `common.py` to map note names to frequencies. Future TODO: Design a more robust detection function that can detect note onsets for repeated notes (i.e. notes with the same/similar frequencies) Co-Authored-By: irinazoccolini <[email protected]>
…cation into create_fx_functions
backend/app/views.py
Outdated
| audio = audio_metadata['audio_samples'] | ||
| sample_rate = audio_metadata['sample_rate'] | ||
| encoded_audio = wav_to_base64(audio, sample_rate) | ||
| # audio_data = filters.change_pitch(audio_data, 3) |
There was a problem hiding this comment.
Is this comment meant to demonstrate how one might implement one of your new filters?
There was a problem hiding this comment.
This was from us testing out the function by playing sound from the webpage. This can be deleted.
There was a problem hiding this comment.
It can be! We were using that line to test out some of the filters using the frontend. We were planning on removing the line (perhaps after the hack session), but we could leave it there for future reference!
…r invalid arguments (With some more debugging)
There was a problem hiding this comment.
This is seriously impressive work, and I'll have to admit that the specifics of some of the filter implementations are beyond my ken. Since I won't be of significant use to you in terms of line-by-line code review, I'll leave a few meta comments:
- The docstring comments and cited research are exceptional. Thanks for considering the needs of the team that'll be coming after you.
- I'm having trouble running most of the test cases locally. Which of them do we expect to be fully functional at this point?
- Is
stretch_audiocomplete, or still a WIP? It seems like it duplicates audio input in sequence rather than 'stretching' it, which seems like a partial implementation that isn't fully complete yet, but I'm willing to admit I might not grok the bigger picture.
Overall the direction is very sound, the code is relatively self-documenting with excellent supplementary docstrings. I'd like to feel confident that the tests support the filter functions, at which point I'll be comfortable approving this. Thanks!
|
The |
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
There was a problem hiding this comment.
Thanks for showing off this work today in the hack session. Just a note for posterity: this PR will contain some code that is still a work in progress. Code comments will point to these places. Otherwise, this work is well tested and very well documented, so I leave my approval. Thanks for all your hard work!
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
The current tests should all be passing. The commented failing tests have some comments that can be used in conjunction with some of the upgrades to the filter functions to make them pass in the future. |
(This makes it easier to filter the audio before sending it to the frontend) Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
Co-Authored-By: irinazoccolini <[email protected]> Co-Authored-By: Ashar Farooq <[email protected]> Co-Authored-By: Watermelanie <[email protected]>
This PR creates filter functions to modify input sound waves for greater flexibility in the audio that is produced from user input. In particular, some functions that this PR will create include:
apply_filter: This is the base function where we call other filter functions for each note.get_notes: This separates the audio into note slices and identifies the root note being played.change_volume: This changes the amplitude of audio samples.stretch_audio: This changes the duration of the audio.change_pitch: This changes the pitch of the notes in the audio.add_chords: This builds major or minor triads upon each note in the audio..