Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ccc #109

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
Open

Ccc #109

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"cmake.configureOnOpen": false
}
23 changes: 23 additions & 0 deletions release-packaging/Classes/FluidConcat.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
FluidConcat : FluidRTUGen {
*ar { arg sourceIn, controlSegmentTrig, controlFeatureTrig, sourceSegmentTrig, sourceFeatureTrig,
controlFeatureBuffer, sourceFeatureBuffer, maxHistoryLength=5000, historyWindowLength=3000, historyWindowOffset=0, fadeTime=100, speed=1, algo=0, randomness=0;
^this.multiNew('audio', sourceIn.asAudioRateInput(this),
controlSegmentTrig.asAudioRateInput(this),
controlFeatureTrig.asAudioRateInput(this),
sourceSegmentTrig.asAudioRateInput(this),
sourceFeatureTrig.asAudioRateInput(this),
controlFeatureBuffer.asUGenInput, sourceFeatureBuffer.asUGenInput,
maxHistoryLength, historyWindowLength, historyWindowOffset, fadeTime, speed, algo, randomness)
}


checkInputs {
// the checks of rates here are in the order of the kr method definition
// if(inputs.at(2).rate != 'scalar') {
//^(": maxNumCoeffs cannot be modulated.");
// };
^this.checkValidInputs;
}

}

13 changes: 13 additions & 0 deletions release-packaging/Classes/FluidETC.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FluidETC : UGen {
*kr { arg in = 0, symbolCount=16, winSize=5, maxWinSize=20, kHopSize=5;
^this.multiNew('control', in, symbolCount, winSize, maxWinSize, kHopSize)
}
checkInputs {
// the checks of rates here are in the order of the kr method definition
if(inputs.at(3).rate != 'scalar') {
^(": maxWinSize cannot be modulated.");
};
^this.checkValidInputs;
}
}

13 changes: 13 additions & 0 deletions release-packaging/Classes/FluidShannonEntropy.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FluidShannonEntropy : FluidRTUGen {
*ar { arg in = 0, symbolCount=1000, winSize=100, maxWinSize=200, kHopSize=50;
^this.multiNew('audio', in.asAudioRateInput(this), symbolCount, winSize, maxWinSize, kHopSize)
}
checkInputs {
// the checks of rates here are in the order of the kr method definition
if(inputs.at(3).rate != 'scalar') {
^(": maxWinSize cannot be modulated.");
};
^this.checkValidInputs;
}

}
16 changes: 16 additions & 0 deletions release-packaging/Classes/FluidShannonEntropyKR.sc
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FluidShannonEntropyKR : FluidRTUGen {
*kr { arg in = 0, symbolCount=1000, winSize=100, maxWinSize=200, kHopSize=50;
^this.multiNew('control', in, symbolCount, winSize, maxWinSize, kHopSize)
}
checkInputs {
// the checks of rates here are in the order of the kr method definition
if(inputs.at(3).rate != 'scalar') {
^(": maxWinSize cannot be modulated.");
};
^this.checkValidInputs;
}

}



132 changes: 132 additions & 0 deletions release-packaging/HelpSource/Classes/FluidConcat.schelp
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
TITLE:: FluidConcat
summary:: Concatenative synthesis with a live database
categories:: Libraries>FluidCorpusManipulation
related:: Concat, Concat2

DESCRIPTION::

FluidConcat is a concatenative synthesiser, that runs from a live signal, using a the recent history of a live signal as a database.
FluidConcat takes data from two signals: a control signal and a source signal. The current frame of the control signal is matched against the recent history of the source signal, and then the closest matching section of the source signal is played back. You can use the same signal for both source and control, in which case the control signal will be matching against its own history.
This UGen integrates with FluCoMa's machine listening UGens: for each sigal you must provide your own slices, and features of those slices.


Inspired by Nick Collin's Concat UGens.

CLASSMETHODS::

METHOD:: ar
(describe method here)

ARGUMENT:: sourceIn
live audio input

ARGUMENT:: controlSegmentTrig
A trigger indicating a new slice in the control signal

ARGUMENT:: controlFeatureTrig
A trigger indicating a new feature of the control signal is avaiable (stored in controlFeatureBuffer)

ARGUMENT:: sourceSegmentTrig
A trigger indicating a new slice in the source signal.

ARGUMENT:: sourceFeatureTrig
A trigger indicating a new feature of the source signal is avaiable (stored in sourceFeatureBuffer)

ARGUMENT:: controlFeatureBuffer
An array of features that describe the slice of audio (e.g. MFCCs, spectral features)

ARGUMENT:: sourceFeatureBuffer
An array of features that describe the slice of audio (e.g. MFCCs, spectral features)

ARGUMENT:: maxHistoryLength (milliseconds)
The maximum length of time for which features of the source signal will be stored

ARGUMENT:: historyWindowLength (milliseconds)
The size of segment of history of the source signal where FluidConcat will search for matches

ARGUMENT:: historyWindowOffset (milliseconds)
The position where the history search window is placed

ARGUMENT:: fadeTime (milliseconds)
The length of fade time between segments being played back

ARGUMENT:: speed
The speed at which segments are played back (1.0 = the same, 0.5 = half speed etc)

ARGUMENT:: algo
The distance algorithm used for matches
0: Euclidean distance
1: Cosine distance

ARGUMENT:: randomness
Level of randomness in the choice of segments to play back. 0 = no randomness, 1=totally random, matching is ignored

returns:: an audio signal


INSTANCEMETHODS::

EXAMPLES::

code::
(some example code)
::
A self-concatenator. Try this with your computer microphone + headphones
(
s.options.memSize_(65536 * 16);
~buf= Buffer.alloc(s, 20, 1);
~arrayToBuffer = {
|ar, buf|
ar.size().do({|i|BufWr.kr([ar[i]], buf, i)});
};

Ndef(\concatSelfConcatTest, {
var feature, input, mfcc, segmentTrig, featureTrig, osc, slice;
feature = LocalBuf.new(10,1);
input = SoundIn.ar([0]);
slice= FluidNoveltySlice.ar(input,1,3,0.5); //adjust this threshold to match the level of the audio input
Poll.ar(slice,slice);
mfcc = FluidMFCC.kr(input, 21, maxNumCoeffs:21, windowSize:256, maxFFTSize:1024);
~arrayToBuffer.(mfcc[1..], ~buf);
segmentTrig=LFPulse.ar(20)-0.5;
featureTrig = LFPulse.ar(SampleRate.ir / 512)-0.5; //the rate of MFCCs being generated
FluidConcat.ar(input, segmentTrig, featureTrig, segmentTrig, featureTrig, ~buf, ~buf, 10000, 5000, 10000, 30, 1, 1, 0.1)!2
}).play
)



A live signal, matching on a database from a Buffer

b = Buffer.read(s,File.realpath(FluidAmpSlice.class.filenameSymbol).dirname.withTrailingSlash ++ "../AudioFiles/Nicol-LoopE-M.wav");
b.play


(
s.options.memSize_(65536 * 16);
~cbuf= Buffer.alloc(s, 20, 1);
~sbuf= Buffer.alloc(s, 20, 1);
~arrayToBuffer = {
|ar, buf|
ar.size().do({|i|BufWr.kr([ar[i]], buf, i)});
};

Ndef(\concatControlTest, {
var feature, input, mfccInput, mfccSource, segmentTrig, featureTrig, osc, sliceControl, sliceSource, loop;
feature = LocalBuf.new(10,1);
input = SoundIn.ar([0]);
loop = PlayBuf.ar(1, b, BufRateScale.kr(b), loop:1);
sliceSource= FluidAmpSlice.ar(loop,fastRampUp: 10,fastRampDown: 2205,slowRampUp: 4410,slowRampDown: 4410,onThreshold: 10,offThreshold: 5,floor: -40,minSliceLength: 4410,highPassFreq: 20);
Poll.ar(sliceSource, sliceSource);
sliceControl= FluidNoveltySlice.ar(input,1,5,0.5);
// Poll.ar(sliceControl, sliceControl);
mfccInput = FluidMFCC.kr(input, 21, maxNumCoeffs:21, windowSize:256, maxFFTSize:1024);
mfccSource = FluidMFCC.kr(loop, 21, maxNumCoeffs:21, windowSize:256, maxFFTSize:1024);
~arrayToBuffer.(mfccInput[1..], ~cbuf);
~arrayToBuffer.(mfccSource[1..], ~sbuf);
featureTrig = LFPulse.ar(SampleRate.ir / 256)-0.5;
FluidConcat.ar(loop, sliceControl, featureTrig, sliceSource, featureTrig, ~cbuf, ~sbuf, 10000, 10000, 10000, 10, 1, 1, 0.0)!2
// osc
}).play
)

66 changes: 66 additions & 0 deletions release-packaging/HelpSource/Classes/FluidETC.schelp
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
TITLE:: FluidETC
summary:: Calculates Effort To Compress (ETC) for an audio signal
categories:: Libraries>FluidCorpusManipulation
related:: FluidShannonEntropy
DESCRIPTION::


*Warning* this UGen makes heavy use of the CPU, and needs plenty of headroom because the algorithm can vary in the number of iterations needed to complete calcuations.
​More info on ETC: https://arxiv.org/abs/1611.00607

Three Perspectives on Complexity − Entropy, Compression, Subsymmetry
Nithin Nagaraj, Karthi Balasubramanian

CLASSMETHODS::
METHOD:: kr
Calculate Effort To Compress for an audio signal

ARGUMENT:: in
An audio signal

ARGUMENT:: symbolCount
The number of discrete symbols to quantise the audio signal to

ARGUMENT:: winSize
The amount of recent history of the audio signal from which to calculate ETC (in milliseconds)

ARGUMENT:: maxWinSize
The maximum size of the window (in milliseconds)

ARGUMENT:: kHopSize
How often to run the calculation (in milliseconds)

returns:: Effort To Compress (between 0 and 1)

INSTANCEMETHODS::
METHOD:: checkInputs
(describe method here)
returns:: (describe returnvalue here)
EXAMPLES::
This example prints the ETC for varying input types, in the post window.

code::
(
{
var sig;
//uncomment to look at the difference between varying types of input
// sig = PinkNoise.ar;
sig = SoundIn.ar([0]);
//sig = SinOsc.ar(2000);
// sig = LFSaw.ar(10).madd(0.5);
FluidETC.kr(sig, 16, 5, 20, 5).poll;
0
}.play
)

::
49 changes: 49 additions & 0 deletions release-packaging/HelpSource/Classes/FluidShannonEntropy.schelp
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
TITLE:: FluidShannonEntropy
summary:: Calculates Shannon Entropy
categories:: Libraries>FluidCorpusManipulation
related:: Guides/FluidCorpusManipulation

DESCRIPTION::

Calculates the Shannon Entropy of the audio input, based on a moving window of recent input, and quantisation of the audio to a number of discrete symbols.

CLASSMETHODS::

METHOD:: ar
Calculate Shannon Entropy of an audio signal

ARGUMENT:: in
An audio signal

ARGUMENT:: symbolCount
The number of discrete symbols to quantise the audio signal to

ARGUMENT:: winSize
The amount of recent history of the audio signal from which to calculate the Shannon Entropy (in milliseconds)

ARGUMENT:: maxWinSize
The maximum size of the window (in milliseconds)

ARGUMENT:: kHopSize
How often to run the calculation (in milliseconds)

returns:: Shannon Entropy



EXAMPLES::

code::
(
{
var sig;
//uncomment to look at the difference between varying types of input
// sig = PinkNoise.ar;
sig = SoundIn.ar([0]);
// sig = SinOsc.ar(2000);
// sig = LFSaw.ar(10);
FluidShannonEntropy.ar(sig, 10000, 100, 500, 50).poll(20)
0
}).play
)
::
50 changes: 50 additions & 0 deletions release-packaging/HelpSource/Classes/FluidShannonEntropyKR.schelp
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
TITLE:: FluidShannonEntropy
summary:: Calculates Shannon Entropy
categories:: Libraries>FluidCorpusManipulation
related:: Guides/FluidCorpusManipulation

DESCRIPTION::

Calculates the Shannon Entropy of the audio input, based on a moving window of recent input, and quantisation of the audio to a number of discrete symbols.
This is the control rate version of FluidShannonEntropy

CLASSMETHODS::

METHOD:: kr
Calculate Shannon Entropy of an audio signal

ARGUMENT:: in
An audio signal

ARGUMENT:: symbolCount
The number of discrete symbols to quantise the audio signal to

ARGUMENT:: winSize
The amount of recent history of the audio signal from which to calculate the Shannon Entropy (in milliseconds)

ARGUMENT:: maxWinSize
The maximum size of the window (in milliseconds)

ARGUMENT:: kHopSize
How often to run the calculation (in milliseconds)

returns:: Shannon Entropy



EXAMPLES::

code::
(
{
var sig;
//uncomment to look at the difference between varying types of input
// sig = PinkNoise.ar;
sig = SoundIn.ar([0]);
// sig = SinOsc.ar(2000);
// sig = LFSaw.ar(10);
FluidShannonEntropy.ar(sig, 10000, 100, 500, 50).poll(20)
0
}).play
)
::
Binary file added release-packaging/plugins/FluidAmpGate.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidAmpSlice.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidAudioTransport.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufAmpGate.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufAmpSlice.so
Binary file not shown.
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufChroma.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufCompose.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufFlatten.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufHPSS.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufLoudness.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufMFCC.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufMelBands.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufNMF.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufNNDSVD.so
Binary file not shown.
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufOnsetSlice.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufPitch.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufSTFT.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufScale.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufSelect.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufSelectEvery.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufSines.so
Binary file not shown.
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufStats.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufThreadDemo.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufThresh.so
Binary file not shown.
Binary file not shown.
Binary file added release-packaging/plugins/FluidBufTransients.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidChroma.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidConcat.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidETC.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidGain.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidHPSS.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidLoudness.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidMFCC.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidManipulation.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidMelBands.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidMessageTest.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidNMFCross.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidNMFFilter.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidNMFMatch.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidNMFMorph.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidNoveltySlice.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidOnsetSlice.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidPitch.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidSTFTPass.so
Binary file not shown.
Binary file not shown.
Binary file added release-packaging/plugins/FluidSines.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidSpectralShape.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidStats.so
Binary file not shown.
Binary file not shown.
Binary file added release-packaging/plugins/FluidTransientSlice.so
Binary file not shown.
Binary file added release-packaging/plugins/FluidTransients.so
Binary file not shown.
21 changes: 21 additions & 0 deletions src/FluidConcat/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Part of the Fluid Corpus Manipulation Project (http://www.flucoma.org/)
# Copyright 2017-2019 University of Huddersfield.
# Licensed under the BSD-3 License.
# See license.md file in the project root for full license information.
# This project has received funding from the European Research Council (ERC)
# under the European Union’s Horizon 2020 research and innovation programme
# (grant agreement No 725899).

cmake_minimum_required(VERSION 3.11)

get_filename_component(PLUGIN ${CMAKE_CURRENT_LIST_DIR} NAME_WE)
message("Configuring ${PLUGIN}")
set(FILENAME ${PLUGIN}.cpp)

add_library(
${PLUGIN}
MODULE
${FILENAME}
)

include(${CMAKE_CURRENT_LIST_DIR}/../../scripts/target_post.cmake)
Loading