FR: let existing Patches register on a new graph with a different output_buffer_size

[balintlaczko]

Hi! I am experiencing jupyter kernel crashing when trying to connect nodes that were played on a previous graph with a different (smaller) output_buffer_size. Minimal example:

# CELL 1
import signalflow as sf
config = sf.AudioGraphConfig()
config.output_buffer_size = 480
graph = sf.AudioGraph(config)

class TestPatch(sf.Patch):
    def __init__(self):
        super().__init__()
        out = sf.SineOscillator(440)
        self.set_output(out)

patch = TestPatch()

graph.play(patch)

Then I want to change the output buffer size:

# CELL 2
graph.destroy()
config = sf.AudioGraphConfig()
config.output_buffer_size = 1024
graph = sf.AudioGraph(config)
print("About to play") # still prints
graph.play(patch) # will crash Kernel here...

Is there a way to somehow "tell" a patch about the new buffer size, or do you always have to re-create the patch from scratch? In my situation, the user may already have a complex patch set up with lots of params set to non-default values, so ideally, it would be great not having to re-create all those from the snapshot of the current state of all params, but I suspect that is the case? Any tips would be much appreciated! :)

# CELL 3
graph.destroy()
config = sf.AudioGraphConfig()
config.output_buffer_size = 1024
graph = sf.AudioGraph(config)
patch = TestPatch() # create a new patch
graph.play(patch) # this will work

[balintlaczko]

Update: after digging through the examples a bit, I found that getting a spec from the Patch and then creating a new one from spec has the benefit of keeping modified param states. So if:

# CELL 1
import signalflow as sf
config = sf.AudioGraphConfig()
config.output_buffer_size = 480
graph = sf.AudioGraph(config)

class TestPatch(sf.Patch):
    def __init__(self):
        super().__init__()
        freq = self.add_input("freq", 440)
        out = sf.SineOscillator(freq)
        self.set_output(out)

patch = TestPatch()

graph.play(patch)

then change the default freq param:

# CELL 2
patch.set_input("freq", 880)

and then convert to spec and re-create from spec:

# CELL 3
graph.stop(patch)
spec = patch.to_spec()
graph.destroy()
config = sf.AudioGraphConfig()
config.output_buffer_size = 1024
graph = sf.AudioGraph(config)
patch2 = sf.Patch(spec)
graph.play(patch2) # this will work and freq will be 880

That is great! But my problem is that I am doing a lot of other things that will not transfer to the new instance this way, plus any synth patches that the user may have in variables will now reference different objects. So consider this an FR:

let existing Patches register on a new graph with a different output_buffer_size

I understand if my problem is none of your business, and I have no idea how deep change this would need to be implemented, but it would help me out a lot! 😄

[ideoforms]

This is a great question and interesting line of investigation. However, I would say that, according to the conceptual model of SignalFlow, a Patch (that is, an instance of a PatchSpec) belongs to a specific AudioGraph, and is not designed to be replayed/reconnected to different graphs. Buffer size is only one issue... it would also need to handle changes in sample rate which would need to propagate down through internal nodes, which certainly doesn't happen at the moment.

The way to approach this problem by design is to indeed create a PatchSpec from the Patch, and then create instances of that PatchSpec on the different graph.

I'd be interested to hear more about the use case and app flow that has led to this though! How come the patches are previously attached to a different graph? Is this to switch between NRT and RT processing?

[ideoforms]

Crashes are something I always want to prevent, however, so I may repurpose this issue to solve the crash when a Patch is played on more than one graph (insteading raising an Exception).

[balintlaczko]

Thanks, that's fair, I'm also thinking of reworking things on my side so that any SignalFlow Patch can be used, and once I have that, I can solve my above-described issue in a more idiomatic way. I think here I wanted to implement an App prop for the buffer size, so that if it is set, then it destroys the graph, changes the buffer size, and makes a new graph, then reconnects all patches. I actually do the same when I switch between RT and NRT, but since I haven't tried to change buffer sizes between those, so far it has worked somehow, i.e. I didn't have to recreate the patches from spec. That's why I originally assumed that it "should be" possible to reconnect when the buffer size (or sample rate) changes as well.

But now I understand that once a Patch object connects to a graph the first time, it cannot/should not connect to any other graph later on.

What I currently have is a "Synth" (that is a Patch with additional stuff), that connects to an App (that holds a reference to the graph). SO something along the lines of:

app = App()
theremin = Theremin() # a patch that also has a UI component and it reads params smoothed from buffers (needed for NRT stuff)
app.attach(theremin) # here the app appends theremin to the list of its synths, and connects it to its Bus' input, so that it will be included in the graph. it also includes a view of the theremin's UI in its own UI
# but I can still do stuff to the theremin since it is stored in a variable, for example I can display its UI:
theremin.ui # which will show another view of the UI (and all views are in sync) where I can change its params with sliders
# or I can set a param
theremin.set_input_buf("frequency", 220)

My problem here is that if it is not allowed to reconnect to another graph, then once a new graph is made in the app (because of NRT or changing buffer size/sample rate)...

app.sample_rate = 44100 # now it has to destroy the old graph and create a new one

...then the Patch will have to be re-created from spec, which means the user loses the option to control it via the reference in the theremin variable. Maybe there is a solution to this, I just have to research it more thoroughly.

[balintlaczko]

...perhaps another FR attempt would be that if a Patch is explicitly removed from the graph via stop() then it could get "ready" to connect to another one like it did the first time?

graph = AudioGraph()
my_patch = Patch(...)
graph.play(my_patch) # now it sets buffer size, sample rate, etc for the Patch
graph.stop(my_patch) # now it is removed, but also, it is "ready" to connect to another graph like the first time

The part that is unclear to me is when the Patch's buffer size and sample rate are set. Is it when the new object is created? But it seems possible to create a Patch before an AudioGraph exists. Or are they set when the Patch for the first time connects to an AudioGraph? In that case there might be a slight chance that the above FR is possible to implement (since the same first-time setup routine could be called again with the new graph). I obviously don't know what I'm talking about, just thinking out loud... :)

[ideoforms]

The part that is unclear to me is when the Patch's buffer size and sample rate are set. Is it when the new object is created? But it seems possible to create a Patch before an AudioGraph exists.

Some of the magic that happens in the background of SignalFlow obscures the true nature of the model. In fact, creating almost any Node/Patch requires that an AudioGraph already exists, and nodes constructors/allocators often make use of the AudioGraph's properties - things like (as you've seen) buffer sizes, and sample rate / sample rate conversion properties. There is no allowance at all made for re-allocating buffers for different graphs, and doing so really would involve a huge amount of work and ongoing complexity and maintenance, as you suspected :)

Just thinking more about your problem. If you don't require that the AudioGraph is generating audio output when NRT rendering takes place, perhaps you could simply stop audio playback temporarily (graph.stop()), render to a buffer from the AudioGraph (buf = AudioGraph.render_to_new_buffer(44100); buf.save("output.wav")), and then restart with graph.start()?

[balintlaczko]

Oh I thought for the NRT I must create the graph using the AudioOut_Dummy as output device, as described here. I didn't even try to do it on the same graph. Okay, if that works, that will be a huge reduction in code complexity. :)

I think I also figured a way to restructure my code to fully adhere to this behavior, by separating all of my code from the Patch and having my Synth only hold a PatchSpec, which can be grabbed to create a new instance any time neccesary (such as on a different graph), while the user could still get all the control and other stuff from the same variable. This will coincidently also solve one of my design limitations regarding polyphony... (: Thanks a lot for the help!

[ideoforms]

Yeah, Dummy is provided as a way for systems that might not even have audio I/O to partake in signal generation.... but you can also get away with just not starting the graph :)

That solution sounds great. Let me know how you get on!