diff --git a/docs/v1.0.0-Arzashkun /1.-Quick-start.md b/docs/v1.0.0-Arzashkun /1.-Quick-start.md new file mode 100644 index 0000000..f077efd --- /dev/null +++ b/docs/v1.0.0-Arzashkun /1.-Quick-start.md @@ -0,0 +1,128 @@ +Let's build your first functional mesh (or just check the full example in [go-playground](https://go.dev/play/p/W48JJlwNfo1)). + + +For simplicity it will be a mesh with just 1 component, which receives single input signal and generates "Hello World!" string as it's output signal. + + + + +### Creating a mesh + +To start, we create a mesh object. Each mesh is uniquely identified by its name, which helps in managing multiple meshes: + +```go +fm := fmesh.New("simple mesh") +``` + +As you can see it has a name, so you can use it when managing multiple meshes in your code. Also there are some other properties like Description or Configuration, but for now we do not need them. + +### Creating a component + +Next, let’s create our first component, ***concat***. This component concatenates two input strings and produces a single output string. + +```go +c := component.New("concat"). //Unique name is required + WithDescription("Concatenates 2 strings"). //Optional + WithInputs("i1", "i2"). //Optional (ports must have unique names) + WithOutputs("o1"). //Optional (ports must have unique names) + WithActivationFunc(func(this *component.Component) error { + //Read input signals + payload1 := this.InputByName("i1").FirstSignalPayloadOrDefault("").(string) + payload2 := this.InputByName("i2").FirstSignalPayloadOrDefault("").(string) + + //Generate output signal + resultSignal := signal.New(payload1 + payload2) + + //Put the signal on port + this.OutputByName("o1").PutSignals(resultSignal) + + //Inform the framework that activation finished successfully + return nil + }) +``` + +The concat component: + +1. Takes inputs from ports ***i1*** and ***i2***. +2. Concatenates them. +3. Outputs the result on port ***o1***. + +In general that is the work of any well encapsulated system (ideally a pure function): read inputs, provide outputs and leave zero side effects. +In F-Mesh you do not _read_ the signal from port, you just _get_ it, because when a component is activated all input signals are already there, buffered on input ports. Same with writes, it is non-blocking operation, that why in F-Mesh we do not _send_, rather we put signals on the ports. This creates a _frozen_ or _discrete_ time experience, where activation functions involve no async I/O, you just take inputs, put outputs and exit with or without error. + +The job done by the component sounds pretty simple and is equivalent of: + +```go +func concat(i1,i2 string) (string, error) { + return i1 + i2, nil +} +``` + +So why use a more complex and verbose approach? Of course, it doesn't make sense to use F-Mesh for simple tasks like concatenating two strings or similar operations. However, when dealing with a component that has tens or even hundreds of input and output ports, that's when F-Mesh truly shines with its declarative syntax! + + +Next, let's add the component to the mesh so it recognizes it + +```go +// Add component to mesh +fm.WithComponents(c) +``` +>[!IMPORTANT] +All components must be explicitly added to the mesh. + +### Passing initial signals + +So far so good, now we can pass some data into the mesh: + +```go +// Pass input signals to mesh +c.InputByName("i1").PutSignals(signal.New("Hello")) +c.InputByName("i2").PutSignals(signal.New(" World!")) +``` + +There is no special API for initialising the mesh, you just put some signals on input ports of any component you want. In real-world scenario you probably will know which components are entry points for the initial data. Putting signals does not trigger anything, it is just like appending an item to slice. + +### Running the mesh + +Now everything is ready to start the computations and run the mesh: + +```go +// Run and check errors +_, err := fm.Run() +if err != nil { + fmt.Println(fmt.Errorf("F-Mesh returned an error: %w", err)) +} +``` + +First, we pass the input signals, and only then do we run the mesh. The mesh is not a long running process by default (though it can be in [more complex use-cases](https://github.com/hovsep/fmesh/blob/main/examples/async_input/main.go)). It behaves more like a computational graph: you build it from components, initialize it with some state (input signals), and then execute it. Running an F-Mesh means triggering activation cycles until a terminal state is reached: an error, a panic, or a natural stop. The "happy path" is the natural stop, which occurs when no component has signals on any input ports. You can learn more about scheduling rules [here](https://github.com/hovsep/fmesh/wiki/Scheduling-rules). + +>[!NOTE] +***Run()*** is a blocking operation — it will block until the mesh reaches a terminal state. As a result of running the mesh, you will receive an activation cycles report and any errors that occurred. The report can be useful for debugging or visualizing the mesh. In the code above, the report is replaced with blank identifiers for simplicity. + +### Getting the results + +```go +// Extract results from mesh +resultPayload := fm.Components().ByName("concat").OutputByName("o1").FirstSignalPayloadOrNil().(string) +fmt.Println("Got from mesh: ", resultPayload) +``` + +There are no surprises here—simply retrieve the signals from the output ports of the respective components (you or the mesh author should know where to place inputs and where to retrieve outputs). And that’s essentially it. If everything is working correctly, you should see something like: + +`Got from mesh: Hello World!` + +By now, we hope you have a sense of how small chunks of data (signals) flow in and out of components. That’s what FBP and F-Mesh are all about: rather than writing imperative code, you describe how your data flows, making it more natural for your task. + +Congratulations! You've just built your first F-Mesh! + +### Chained API and error handling + +The framework utilizes a chained API to minimize repetitive error checks, allowing you to focus on actual programming. You only need to check for errors once at the end of the chain: + +```go +if sig := c.Inputs().ByName("invalid port name").Buffer().First(); sig.HasErr() { + // Handle error +} +``` + +All main APIs have ***HasErr()*** and ***Err()*** methods, which can be used to check if an error occurred in the chain of calls. Errors are propagated up the chain to the parent level (e.g., signal → signal group/collection → port → port group/collection → component → components collection → F-Mesh). \ No newline at end of file diff --git a/docs/v1.0.0-Arzashkun /2.-Signals.md b/docs/v1.0.0-Arzashkun /2.-Signals.md new file mode 100644 index 0000000..31796d8 --- /dev/null +++ b/docs/v1.0.0-Arzashkun /2.-Signals.md @@ -0,0 +1,40 @@ +## Overview + +In F-Mesh, the data exchanged between components is represented as **signals** — flexible units of information that can carry any type of payload. Signals serve as the foundation for communication within the mesh, enabling the dynamic flow of data between components. Each signal encapsulates exactly one piece of data, referred to as its **payload**. + +To explore the full capabilities of signals, refer to the [signal package API](https://pkg.go.dev/github.com/hovsep/fmesh/signal). + +## Payload + +The payload of a signal can be any valid Go data type, offering immense flexibility. This includes primitive types like integers and strings, complex types like structs and maps, or even nil. + +It’s important to note that **every signal in F-Mesh always has a payload**, even if that payload is nil. There is no concept of an "empty" signal, ensuring consistency in data handling across the mesh. + +Example of creating a simple signal: + +```go +mySignal := signal.New("example payload") // A signal with a string payload +``` + +## Signal Groups + +While individual signals are useful for most cases, there are scenarios where working with a **group of signals** simplifies the design. A signal group aggregates multiple signals, each potentially carrying a different type of payload. + +This flexibility allows components to process diverse datasets in a unified manner. For instance, you could group signals with an integer, a **nil**, a slice, and a map payload, and handle them collectively. + +## Creating a Signal Group + +Here’s an example of creating a signal group with mixed payload types: + +```go +mySignals := signal.NewGroup(1, nil, 3, []int{4, 5, 6}, map[byte]byte{7: 8}) // Group of 5 signals +``` + +In this example the group contains five signals with the following payloads: +1. An integer (1) +2. A nil payload +3. Another integer (3) +4. A slice of integers ([]int{4, 5, 6}) +5. A map of byte-to-byte values (map[byte]byte{7: 8}) + +This versatility makes signal groups a powerful tool for managing complex data flows in F-Mesh. \ No newline at end of file diff --git a/docs/v1.0.0-Arzashkun /3.-Ports.md b/docs/v1.0.0-Arzashkun /3.-Ports.md new file mode 100644 index 0000000..4a5a166 --- /dev/null +++ b/docs/v1.0.0-Arzashkun /3.-Ports.md @@ -0,0 +1,45 @@ +## Overview + +Ports are the connection points of components in F-Mesh. Unlike systems that follow the actor model (e.g., Akka), where components communicate directly, F-Mesh components are isolated and communicate only through signals passed via ports. + +This design promotes modularity and separation of concerns: a component only interacts with its own ports and does not need to know where its signals are routed. The abstraction ensures that the logic and behavior of each component remain self-contained and decoupled from the overall mesh structure. + + +You can explore the full API for ports [here](https://pkg.go.dev/github.com/hovsep/fmesh/port). + +## Types of Ports + +There are two types of ports in F-Mesh: + + 1. **Input Ports**: Receive signals for the component to process. + 2. **Output Ports**: Send signals to connected components. + +Both input and output ports are instances of the same underlying type, [Port](https://github.com/hovsep/fmesh/blob/main/port/port.go#L16). +However, F-Mesh enforces type-safe connections: + +* Pipes cannot be created between two input ports or two output ports. +* Connections are strictly from an output port to an input port. + +## Internal Structure + +Each port has three core elements: + + 1. **Name**: Unique name used to access port. + 2. **Signal Buffer**: Holds an unlimited number of signals. + 3. **Pipes List**: Tracks outbound connections to other ports. + +>[!IMPORTANT] +The port buffer does not guarantee any specific order for the signals it holds. +Your program must not rely on the observed order of signals. + +## Working with Groups and Collections + +When working with multiple ports, F-Mesh provides two utility types: Group and Collection. These abstractions simplify interactions with multiple ports, although users typically do not need to interact with them directly. + +## Group + +A [Group](https://github.com/hovsep/fmesh/blob/main/port/group.go#L13) is essentially a wrapper around a slice of ports. It allows you to iterate over and manage a set of ports without concern for their individual names. + +## Collection + +A [Collection](https://github.com/hovsep/fmesh/blob/main/port/collection.go#L15) is an indexed structure, where ports are stored and accessed by name. Unlike a Group, a Collection cannot contain multiple ports with the same name, ensuring a unique mapping of port names to ports. \ No newline at end of file diff --git a/docs/v1.0.0-Arzashkun /4.-Piping.md b/docs/v1.0.0-Arzashkun /4.-Piping.md new file mode 100644 index 0000000..544ffbb --- /dev/null +++ b/docs/v1.0.0-Arzashkun /4.-Piping.md @@ -0,0 +1,106 @@ +A mesh with only one component is rarely useful. The true power of FMesh emerges when multiple components are connected using **pipes**. By connecting components, you enable them to exchange signals and tackle complex problems in small, manageable steps. This idea is not unique to FMesh—it draws inspiration from concepts in object-oriented and functional programming. However, FMesh implements it in a simpler, more transparent way. + +Pipes are intentionally simple: they merely connect two ports. In fact, "pipe" is an abstraction—there is no dedicated "pipe" entity in the source code. Instead, pipes are represented as groups of outbound [ports](https://github.com/hovsep/fmesh/blob/main/port/port.go#L21). Each pipe connects exactly one output port to exactly one input port. While we initially explored more flexible "any-to-any" pipes, this approach introduced unnecessary complexity and was ultimately avoided. + +This design doesn't limit your ability to create ***one-to-many*** or ***many-to-one*** connections. Such configurations are achieved using multiple pipes, each maintaining the simplicity of a single output-to-input link. + +## One to one connection + + +![](https://github.com/user-attachments/assets/c833488a-5d39-4624-8410-f068c10c8d26) + +To demonstrate how pipes work, let’s start with a simple example: + +```go +//Connect output port "o1" of component "c1" to input port "i1" of component "c2" +c1.OutputByName("o1").PipeTo( + c2.InputByName("i1"), + ) +``` +**Semantics**: All signals put to the output port ***o1*** of ***c1*** will be transferred to the input port ***i1*** of ***c2***. + +>[!IMPORTANT] +Signals are always copied by reference in FMesh. Instead of creating physical copies, FMesh moves pointers between ports. This means signals are removed from the source port's buffer and appended to the destination port's buffer. + +## One-to-Many Connections + +![](https://github.com/user-attachments/assets/09f18fe8-a3eb-444a-b6e9-08fc7ce658ed) + +To create one to many connection we create multiple pipes from the same source port: + +```go +//Create three pipes from "o1" +c1.OutputByName("o1").PipeTo( + c2.InputByName("i1"), + c3.InputByName("i2"), + c4.InputByName("i3"), + ) +``` + +**Semantics**: All signals put on ***o1*** will be copied by reference to all three destination ports (***i1***,***i2*** and ***i3***). + +>[!IMPORTANT] +>Signals are always copied by reference. To avoid unexpected behavior: +>* Keep signals immutable whenever possible. +>* Design components as pure functions to minimize side effects. +>* If necessary, create a deep copy of the signal before modifying it. + +Understanding how pointers work in Go is crucial for working with FMesh pipes effectively + +## Many-to-One Connections + +![](https://github.com/user-attachments/assets/2d367069-f267-4d70-9064-c60162500720) + +Creating a **many-to-one** connection is equally straightforward using the same API: + +```go +// Connect multiple source ports to the same destination port +c1.OutputByName("o1").PipeTo(sink.InputByName("i1")) +c2.OutputByName("o1").PipeTo(sink.InputByName("i1")) +c3.OutputByName("o1").PipeTo(sink.InputByName("i1")) +``` + +**Semantics**: This configuration creates three pipes from the output ports of different components (***c1***, ***c2***, and ***c3***) to the same input port (***i1***) of the ***sink*** component. All signals from the respective source ports will appear on ***i1*** + +You can find similar examples in [this integration test](https://github.com/hovsep/fmesh/blob/main/integration_tests/piping/fan_test.go). + +## Cyclic Connections + +![](https://github.com/user-attachments/assets/7bec7fc4-12ec-4583-99ec-611d4a7ad88d) + +In some scenarios, you may need to create a **cyclic connection**, where a component’s output port is connected to one of its own input ports. This is fully supported and a common pattern in FMesh. Just as functional programming favors recursion over loops, cyclic pipes enable components to "self-activate." + +Example: + +```go +// Connect the output port "o1" to the input port "i1" within the same component +c1.OutputByName("o1").PipeTo(c1.InputByName("i1")) +``` + +**Semantics**: The component will reactivate itself in the next cycle, provided there is at least one signal on any of its input ports. This allows the component to control when to activate and how many cycles to execute before stopping + +For a practical example, check out [the Fibonacci example](https://github.com/hovsep/fmesh/blob/main/examples/fibonacci/main.go#L40), which demonstrates how cyclic pipes can be used to implement recursive logic. + +That’s all you need to know about pipes in FMesh. Pipes are the backbone of communication between components, enabling you to build modular, efficient, and highly flexible systems. With the fundamental building blocks of ports and pipes, you can create a wide range of powerful and versatile patterns, including: + +* **Event Bus**: Centralized communication for decoupled components. +* **Fan-In**: Merging multiple inputs into a single unified output. +* **Fan-Out**: Distributing a single input to multiple destinations. +* **Broadcast**: Simultaneously transmitting a signal to all connected receivers. +* **Chain of Responsibility**: Passing signals through a series of components, each capable of handling or forwarding them. +* **Load Balancer**: Distribute incoming signals across multiple components to balance workload. +* **Round-Robin Distributor**: Sequentially route signals to multiple components in a cyclical order. +* **Pipeline**: Process data through a series of components, each performing a distinct transformation or operation. +* **Aggregator**: Collect signals from multiple sources and combine them into a single output. +* **Filter**: Route signals selectively based on specific criteria or conditions. +* **Splitter**: Divide a signal into smaller parts and distribute them to different components for parallel processing. +* **Priority Queue**: Process signals based on their priority levels, ensuring high-priority tasks are handled first. +* **Dead Letter Queue**: Capture signals that cannot be processed by any component for later analysis or retries. +* **Observer Pattern**: Notify multiple components of changes in a shared signal or state. +* **State Machine**: Use cyclic connections to implement state transitions driven by signals. +* **Pub-Sub (Publish-Subscribe)**: Allow components to subscribe to specific topics or signals and receive updates dynamically. +* **Circuit Breaker**: Monitor signals and temporarily halt processing when a failure threshold is reached. +* **Rate Limiter**: Throttle the flow of signals to ensure components are not overwhelmed. +* **Retry Logic**: Automatically reprocess failed signals after a specified delay or condition. + +These are just a few examples. The simplicity and flexibility of FMesh make it possible to design and implement countless other patterns tailored to your specific needs. \ No newline at end of file diff --git a/docs/v1.0.0-Arzashkun /5.-Component.md b/docs/v1.0.0-Arzashkun /5.-Component.md new file mode 100644 index 0000000..48d27af --- /dev/null +++ b/docs/v1.0.0-Arzashkun /5.-Component.md @@ -0,0 +1,261 @@ +## Overview +Components are the primary building blocks in F-Mesh, encapsulating a unit of behavior in a single function known as the **Activation Function**. +This design ensures that components remain lightweight and transient, activating only when needed and completing execution promptly. +Unlike traditional FBP systems, F-Mesh components are not long-running processes like goroutines; +instead, they execute their logic and return control to the scheduler after each cycle. + +F-Mesh components are generally stateless by default, but they now have the capability to maintain state between activations. +Each component contains its own **State**—a thread-safe key-value store that persists data across cycles. +This allows for more complex and persistent behavior without sacrificing the modularity, predictability, and reusability of the design. +The activation function is invoked every time F-Mesh schedules the component. +If the component fails to complete its execution promptly, it can cause the entire mesh to hang, so it's critical to design components to handle small, focused tasks: +read input signals, process them, and produce output signals, all within the designated cycle. + +## Key Elements of a Component + +1. **Name** +Must be unique within a single mesh. +Used for identification and debugging. + +2. **Inputs** and **Outputs** +A collection of output ports where processed signals are sent. +Components can have an unlimited number of output ports. + +4. **Activation Function** +Defines the logic of the component. +Executes when the component is scheduled. + +5. **State** +Simple key-value storage which can be used to persist data between activation cycles. +You can initialize a component state calling **WithInitialState** when building it. + +6. **Logger** +Each component is instrumented with a simple ***log.Logger** from standard library. +Later we can switch to something fancy like Uber Zap. +The logger has set the component name as a prefix. + +You can optionally provide a description for a component. This description can be useful for visualization when exporting or documenting a mesh. + +>[!NOTE] +While components can have many ports, you are not required to use all of them in every activation. +The usage of ports depends entirely on your component's design and logic. + +To explore the full capabilities of signals, refer to the [component package API](https://pkg.go.dev/github.com/hovsep/fmesh/component). + +## Activation function +The Activation Function is the heart of every component, defining its core behavior. To design elegant and maintainable systems, always strive to keep your activation functions simple and concise. + +All activation functions in F-Mesh share the same signature: +```go +func(this *component.Components) error +``` +This signature allows you to access everything you need, like: + +* **this.Inputs()**: Collection of input ports. +* **this.Outputs()**: Collection of output ports. +* **this.State()**: Current state. +* **this.Logger()**: Logger. + +The function returns an error to indicate if the activation encountered any issues. + +Both inputs and outputs are of type port.Collection, allowing you to use the same [API](https://pkg.go.dev/github.com/hovsep/fmesh/port) to interact with ports. Typically, you will: + +* Read signals from input ports. +* Process these signals. +* Optionally read from or write to state. +* Write results as signals to output ports. + +When used properly the pattern resembles the concept of pure functions in functional programming, promoting clean and predictable behavior. + +**Example: Summing Input Signals** + +Here’s a straightforward activation function that demonstrates how to use input and output ports: + +```go +func(this *component.Component) error { + sum := 0 + + // Access all signals from input port "i1" + for _, sig := range this.InputByName("i1").AllSignalsOrNil() { + sum += sig.PayloadOrNil().(int) // Extract and type-cast the payload + } + + // Create a new signal with the sum and put it on output port "o1" + this.OutputByName("o1").PutSignals(signal.New(sum)) + + return nil // Return nil to indicate success +} +``` + +Explanation: + +* The function calculates the sum of all integer payloads received on input port ***i1***. +* A new signal with the computed sum is sent to output port ***o1***. +* The loop iterates over all signals in the input port's buffer, allowing flexible handling of multiple signals. + +## Stateful component + +As it is mentioned earlier components are stateless by default, but when you need you can make them stateful which means you can preserve some state between activation function invocations. +Check the [State](https://pkg.go.dev/github.com/hovsep/fmesh@v1.0.0-Arzashkun/component#State) type for all available methods. + +Here is an example of **stateful counter**: + +```go + counter := component.New("stateful_counter"). + WithDescription("counts all observed signals and bypasses them down the stream"). + WithInputs("bypass_in"). + WithOutputs("bypass_out"). + WithInitialState(func(state component.State) { + // Explicitly initialize state with the key we want to use (state is just wrapper around map[string]any) + state.Set("observed_signals_count", 0) + }). + WithActivationFunc(func(this *component.Component) error { + // Read the current value from state + count := this.State().Get("observed_signals_count").(int) + + defer func() { + // Once we finished activation we want to write back (persist) our state + this.State().Set("observed_signals_count", count) + }() + + count += this.InputByName("bypass_in").Buffer().Len() + this.Logger().Println("so far signals observed ", count) + + _ = port.ForwardSignals(this.InputByName("bypass_in"), this.OutputByName("bypass_out")) + return nil + }) +``` + + +## Returning Errors + +In most cases, your activation function will complete successfully, and you can simply return ***nil***. However, if an issue arises, returning an error is the proper way to communicate it to F-Mesh. Here's how error handling works: + + * **Error Propagation**: When your activation function returns an error, it notifies F-Mesh of the problem. + * **Mesh Behavior**: Returning an error does not halt the entire mesh unless the error-handling strategy is explicitly set to ***StopOnFirstErrorOrPanic***. This allows your mesh to continue processing other components even if one encounters an issue. + * **Error Handling Strategy**: + If you expect components to occasionally fail and want the mesh to proceed regardless, choose an error-handling strategy that tolerates errors when creating the mesh. + Examples of such strategies include logging errors or collecting them for later inspection without stopping execution. + + * **Signal Flushing**: Errors do not affect how signals are drained from output ports. Any signals that were added to the output ports before the error occurred will still be flushed as usual. + * **Inspecting errors after execution**: When you call **fm.Run()**, it provides detailed information about errors encountered during execution. The first return value contains the completed cycles, while the second return value is an error with running fmesh itself. This allows you to review the specific components activation results and understand the nature of the errors for debugging or reporting purposes. +Here is the struct you will get per each component activation: + +```go +type ActivationResult struct { + *common.Chainable + componentName string //The name of the component + activated bool // Did it activate? (e.g. if component was not "scheduled" in given cycle you will see false here) + code ActivationResultCode // The code describing what happened with the component from FMesh point of view, see codes below + activationError error //Error returned from component activation function +} +``` + +And here is the description of activation result codes: + +- **`ActivationCodeUndefined`**: The component's state is not defined. +- **`ActivationCodeOK`**: The activation function executed successfully. +- **`ActivationCodeNoInput`**: The component does not have any input signals. +- **`ActivationCodeNoFunction`**: No activation function is assigned to the component. +- **`ActivationCodeReturnedError`**: The activation function encountered an error and returned it. +- **`ActivationCodePanicked`**: The activation function caused a panic during execution. +- **`ActivationCodeWaitingForInputsClear`**: The component is waiting for input signals on particular ports and decided to clear its current inputs. +- **`ActivationCodeWaitingForInputsKeep`**: The component is waiting for input signals on particular ports and decided to keep all current inputs till the next cycle. + + +### Example + +Here’s a simple example that demonstrates returning an error: + +```go +func(this *component.Component) error { + // This signal will be successfully transferred, as it is put before any error is returned + this.OutputByName("log").PutSignals(signal.New("component activated")) + + firstPayload := this.InputByName("i1").FirstSignalPayloadOrNil() + if firstPayload == nil { + return fmt.Errorf("no signals received on input port 'i1'") + } + + number, ok := firstPayload.(int) + if !ok { + return fmt.Errorf("expected integer payload on 'i1', but got %T", firstPayload) + } + + this.OutputByName("o1").PutSignals(signal.New(number * 2)) + return nil // Success +} +``` +Explanation: + + * If no signals are received on the input port ***i1***, an error is returned with a descriptive message. + * If the payload type is not an integer, an error is returned indicating the type mismatch. + * If everything is fine, the function processes the input and sends a signal to the output port ***o1*** with the doubled value. + +Key Considerations + + * Use meaningful error messages to help diagnose issues during execution. + * Ensure your mesh's error-handling strategy aligns with your system's requirements. + * Errors are a tool to maintain clarity and predictability in your system without disrupting the entire flow unnecessarily. + +## Waiting for inputs + +In some cases, you may need to delay activation of a component until specific signals appear on one or more of its ports. F-Mesh provides a basic synchronization mechanism for such scenarios, allowing you to return a special error from the activation function to signal that the component should wait. + +Let’s examine the following mesh setup: + + + +### Initializing the Mesh + +Here’s how we initialize the mesh with input signals: + +```go +// Put one signal into each chain to start them in the same cycle +fm.Components().ByName("d1").InputByName("i1").PutSignals(signal.New(1)) +fm.Components().ByName("d4").InputByName("i1").PutSignals(signal.New(2)) +``` + +This configuration starts execution at the topmost components (d1 and d4) and progresses downward in parallel. The activation cycles will look like this: + + 1. **d4**, **d1** + 2. **d5**, **d2** + 3. **sum**, **d3** + 4. **sum** + +### The Synchronization Problem + +Suppose the sum component needs to compute the **sum** of the signals from both vertical chains. A problem arises at cycle #3 because the left chain is shorter, causing its signal to arrive at **sum** earlier. To resolve this, we can instruct F-Mesh to wait until both input ports of **sum** have signals before activating it. + +### Implementation + +Here’s how to implement this behavior: +```go +s := component.New("sum").WithDescription("This component just sums 2 inputs"). + WithInputs("i1", "i2"). + WithOutputs("o1"). + WithActivationFunc(func(this *component.Component) error { + // Wait until both input ports have signals + if !this.Inputs().ByNames("i1", "i2").AllHaveSignals() { + return component.NewErrWaitForInputs(true) + } + + inputNum1 := this.InputByName("i1").FirstSignalPayloadOrDefault(0) + inputNum2 := this.InputByName("i2").FirstSignalPayloadOrDefault(0) + + this.OutputByName("o1").PutSignals(signal.New(inputNum1.(int) + inputNum2.(int))) + return nil + }) +``` +The critical part of the implementation is the use of: + +```go +component.NewErrWaitForInputs(true) +``` + +The boolean flag passed here determines whether to preserve or clear the input ports' buffers while waiting: + + * **true**: Keeps all signals in the input buffers untouched. This is ideal when every signal is important, allowing you to collect multiple signals on each port (remember, ports can buffer an unlimited number of signals). + * **false**: Clears the input buffers while waiting. This mode is suitable when the presence of signals on specific ports matters more than the actual content of the signals. + +By using this mechanism, you can control when a component should activate and ensure proper synchronization in your mesh. \ No newline at end of file diff --git a/docs/v1.0.0-Arzashkun /6.-Scheduling-rules.md b/docs/v1.0.0-Arzashkun /6.-Scheduling-rules.md new file mode 100644 index 0000000..7124fba --- /dev/null +++ b/docs/v1.0.0-Arzashkun /6.-Scheduling-rules.md @@ -0,0 +1,84 @@ +You can think of FMesh as an orchestrator or computational graph, designed to run components—whether all of them or just a select few—based on a structured approach. Execution cannot happen arbitrarily; it must follow specific scheduling rules. These rules ensure that components are activated in the correct order, maintaining the integrity and efficiency of your mesh. Understanding how FMesh schedules components is crucial for building a well-functioning and reliable system. + +Execution in FMesh is organized into **Activation Cycles**, which represent the process of activating a group of components. Each activation cycle is structured into distinct phases to ensure proper execution: + +## Phases of an Activation Cycle + +1. **Determining Scheduled Components** + * In this phase, FMesh identifies all components that are ready to be activated. + * A component is considered ready if at least one of its input ports contains signals (i.e., is not empty). + * All components identified during this phase are considered **scheduled for activation**. + +2. **Activating Scheduled Components** + * All scheduled components are activated concurrently, with each running in a separate goroutine. + * This concurrency is safe and efficient since components in FMesh are designed to avoid shared state. + * Activation doesn’t always guarantee the invocation of a component's activation function. In some cases, activation may fail before reaching the function (e.g., due to a chained error). + * Regardless of the outcome, FMesh records the activation result, which could include: + + * **Normal Execution**: The activation function completes successfully. + * **Error Return**: The function returns an error. + * **Panic**: The function panics. + + The mesh itself remains robust and does not crash due to individual component failures. Depending on the configuration, the mesh may stop or continue execution. + +3. **Draining Phase** + * Once all scheduled components are activated, the draining phase begins: + * **Input Ports**: All input ports of activated components are cleared, as their signals have been processed and are no longer needed. + * **Output Ports**: Signals residing on the output ports of activated components are flushed into their respective pipes. These signals are then delivered to the input ports at the other ends of the pipes. This ensures that signals are effectively transferred and ready for the next cycle. + +**Special case**: If a component is in a ["waiting"](https://github.com/hovsep/fmesh/wiki/5.-Component#waiting-for-inputs) state (expecting more input), its input ports may not be cleared. +Additionally, the component’s output ports are never flushed while it is waiting. The exact behavior is determined by the component's implementation. + +After completing the draining phase, FMesh proceeds to the next activation cycle, repeating the process. This iterative execution continues until the mesh reaches a terminal state, at which point processing concludes. + + +## Terminal States in FMesh + +FMesh execution concludes when the mesh reaches a terminal state. The following scenarios define these terminal states: + +* **Chained Error Propagation** +A chained error is propagated to the mesh, indicating that a critical issue occurred at the signal, port, or component level, making further execution invalid. + +* **Cycle Limit Reached** +FMesh can be configured to terminate after a specified number of activation cycles. This is particularly useful for debugging or testing purposes. Example configuration: + +```go + fm := fmesh.New("limited mesh").WithConfig(fmesh.Config{ + CyclesLimit: 10, // Limit to 10 activation cycles + }) +``` + +* **No Components Activated in the Last Cycle** +If no components are activated during an activation cycle, it signifies that all signals have been processed, and the mesh has completed its execution naturally. At this point, it is time to extract the final results. + +* **Error Handling Strategy: StopOnFirstErrorOrPanic** +If the error handling strategy is set to StopOnFirstErrorOrPanic, the mesh will terminate as soon as any component encounters an error or panic during activation. + +* **Error Handling Strategy: StopOnFirstPanic** +If the error handling strategy is configured as StopOnFirstPanic, the mesh will halt immediately upon encountering a panic in any component, ignoring other types of errors. + + +Error handling strategy can be set as follows: +```go +fm := fmesh.New("durable").WithConfig(fmesh.Config{ + ErrorHandlingStrategy: fmesh.IgnoreAll, //Errors and panics will be tolerated + }) +``` + +## Error Handling Strategies in FMesh + +FMesh provides flexible error handling strategies to control how the mesh responds to errors and panics during component activation. The available strategies are: + +* **StopOnFirstErrorOrPanic** +The mesh stops execution as soon as an error or panic occurs in any component's activation function. +This strategy ensures that any unexpected or invalid state is caught early, preventing further processing. + +* **StopOnFirstPanic** +The mesh ignores errors but halts immediately if a panic occurs during a component's activation. +This strategy is useful when errors are recoverable but panics indicate critical failures that require immediate attention. + +* **IgnoreAll** +The mesh continues running regardless of whether components finish their activation functions with errors or panics. +This strategy is ideal for scenarios where robustness is prioritized, and individual component failures do not impact overall processing. + +Each strategy offers different levels of fault tolerance and control, allowing you to tailor the mesh's behavior to suit your application's requirements. diff --git a/docs/v1.0.0-Arzashkun /7.-Export.md b/docs/v1.0.0-Arzashkun /7.-Export.md new file mode 100644 index 0000000..e4beb2b --- /dev/null +++ b/docs/v1.0.0-Arzashkun /7.-Export.md @@ -0,0 +1,82 @@ +## Exporting Your Mesh + +F-Mesh includes a powerful feature: the ability to visualize your program. Similar to other Flow-Based Programming (FBP) systems, F-Mesh operates with standard building blocks—components—which makes it naturally suited for representation in both visual and structural formats (e.g., JSON). + +Currently, the latest release supports one export format: [DOT](https://graphviz.org/doc/info/lang.html). DOT, designed for graph representation, is an ideal fit for F-Mesh's structure. If needed, you can easily add your own export format by implementing the [Exporter](https://github.com/hovsep/fmesh/blob/main/export/exporter.go#L9) interface: + +```go +// Exporter is the common interface for all formats +type Exporter interface { + // Export returns the F-Mesh structure in some format + Export(fm *fmesh.FMesh) ([]byte, error) + + // ExportWithCycles returns the F-Mesh state for each activation cycle + ExportWithCycles(fm *fmesh.FMesh, activationCycles cycle.Cycles) ([][]byte, error) +} +``` + +This interface is straightforward: you receive the mesh and return its representation. The **Export** method represents the static structure of the mesh, while **ExportWithCycles** provides a dynamic view, showing the state of the mesh at each activation cycle—useful for debugging and visualizing the execution process. + +Check out the [export](https://pkg.go.dev/github.com/hovsep/fmesh/export) and [dot](https://pkg.go.dev/github.com/hovsep/fmesh/export/dot) package documentation for more details. + + +## Using an Exporter + +Let’s demonstrate how to export a mesh using the [DOT Exporter](https://pkg.go.dev/github.com/hovsep/fmesh/export/dot). We'll use [this example](https://github.com/hovsep/fmesh/blob/main/integration_tests/ports/waiting_for_inputs_test.go#L92) for demonstration: + +```go +// Create fm +// ... +exporter := dot.NewDotExporter() +data, err := exporter.Export(fm) +if err != nil { + panic("failed to export mesh") +} + +os.WriteFile("graph.dot",data, 0755) +``` +If everything is successful, the graph.dot file will contain the DOT representation: + +```dot +digraph { + layout="dot";splines="ortho"; + + subgraph cluster_7 { + cluster="true";color="black";label="d5";margin="20";penwidth="5";style="rounded"; +... +``` +You can now visualize the mesh using tools like [Edotor.net](https://edotor.net/) or render it with [Graphviz](https://graphviz.org/doc/info/command.html): + +```bash +cat graph.dot | dot -Tpng > graph.png +``` + + + +>[!TIP] +You can customize every aspect of the graph's rendering by using **NewDotExporterWithConfig** + +Graphviz supports various output formats such as PNG, SVG, and PDF. See the full list of supported formats [here](https://graphviz.org/docs/outputs/). + +## Exporting Mesh with Cycles + +To export a mesh along with its activation cycles, pass the cycle data to the exporter and save each cycle separately: +```go +cycles, err := fm.Run() +exporter := dot.NewDotExporter() +data, err := exporter.ExportWithCycles(fm, cycles) +if err != nil { + panic("failed to export mesh") +} + +for cycleNumber, cycleGraph := range data { + os.WriteFile(fmt.Sprintf("cycle-%d.dot", cycleNumber),cycleGraph, 0755) +} +``` +This code creates a separate .dot file for each cycle (e.g., cycle-0.dot, cycle-1.dot). You can use these files to create an animation of your program's execution, such as a GIF: + +![](https://github.com/user-attachments/assets/3ac501e7-b62f-4fd6-9908-be399a6ca464) + +>[!NOTE] +>* On Cycle-3, the layout changes because the **sum** component is waiting for inputs. +>* In the final cycle, no components are executed, and the mesh finishes naturally. diff --git a/docs/v1.0.0-Arzashkun /Home.md b/docs/v1.0.0-Arzashkun /Home.md new file mode 100644 index 0000000..a67e187 --- /dev/null +++ b/docs/v1.0.0-Arzashkun /Home.md @@ -0,0 +1,28 @@ +F-Mesh is a Go-based framework inspired by Flow-Based Programming (FBP) that enables the creation of data flow networks using interconnected components, each processing signals. Components may have multiple input and output called "ports" linked via type-agnostic pipes. + +# Installation + +``` +go get github.com/hovsep/fmesh +``` + +# Release naming convention + +F-Mesh releases are named after 17 historical capitals of Armenia, honouring the ancient cities that played foundational roles in Armenian history. This tradition highlights the project's growth with each version, paralleling Armenia's own historical progression. + + +# Dependencies + +Latest release has exactly one dependency used only for unit tests (which is kinda cool): + +``` +github.com/stretchr/testify +``` + +# API reference +* [FMesh](https://pkg.go.dev/github.com/hovsep/fmesh) +* [Component](https://pkg.go.dev/github.com/hovsep/fmesh/component) +* [Port](https://pkg.go.dev/github.com/hovsep/fmesh/port) +* [Signal](https://pkg.go.dev/github.com/hovsep/fmesh/signal) +* [Export](https://pkg.go.dev/github.com/hovsep/fmesh/export) +* [Common](https://pkg.go.dev/github.com/hovsep/fmesh/common) \ No newline at end of file