-
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
8 changed files
with
774 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
Let's build your first functional mesh (or just check the full example in [go-playground](https://go.dev/play/p/W48JJlwNfo1)). | ||
|
||
|
||
For simplicity it will be a mesh with just 1 component, which receives single input signal and generates "Hello World!" string as it's output signal. | ||
|
||
<img src="https://github.com/user-attachments/assets/989f9be7-9345-48d4-a21a-6e0b17a6a09b" width="60%" /> | ||
|
||
|
||
### Creating a mesh | ||
|
||
To start, we create a mesh object. Each mesh is uniquely identified by its name, which helps in managing multiple meshes: | ||
|
||
```go | ||
fm := fmesh.New("simple mesh") | ||
``` | ||
|
||
As you can see it has a name, so you can use it when managing multiple meshes in your code. Also there are some other properties like Description or Configuration, but for now we do not need them. | ||
|
||
### Creating a component | ||
|
||
Next, let’s create our first component, ***concat***. This component concatenates two input strings and produces a single output string. | ||
|
||
```go | ||
c := component.New("concat"). //Unique name is required | ||
WithDescription("Concatenates 2 strings"). //Optional | ||
WithInputs("i1", "i2"). //Optional (ports must have unique names) | ||
WithOutputs("o1"). //Optional (ports must have unique names) | ||
WithActivationFunc(func(this *component.Component) error { | ||
//Read input signals | ||
payload1 := this.InputByName("i1").FirstSignalPayloadOrDefault("").(string) | ||
payload2 := this.InputByName("i2").FirstSignalPayloadOrDefault("").(string) | ||
|
||
//Generate output signal | ||
resultSignal := signal.New(payload1 + payload2) | ||
|
||
//Put the signal on port | ||
this.OutputByName("o1").PutSignals(resultSignal) | ||
|
||
//Inform the framework that activation finished successfully | ||
return nil | ||
}) | ||
``` | ||
|
||
The concat component: | ||
|
||
1. Takes inputs from ports ***i1*** and ***i2***. | ||
2. Concatenates them. | ||
3. Outputs the result on port ***o1***. | ||
|
||
In general that is the work of any well encapsulated system (ideally a pure function): read inputs, provide outputs and leave zero side effects. | ||
In F-Mesh you do not _read_ the signal from port, you just _get_ it, because when a component is activated all input signals are already there, buffered on input ports. Same with writes, it is non-blocking operation, that why in F-Mesh we do not _send_, rather we put signals on the ports. This creates a _frozen_ or _discrete_ time experience, where activation functions involve no async I/O, you just take inputs, put outputs and exit with or without error. | ||
|
||
The job done by the component sounds pretty simple and is equivalent of: | ||
|
||
```go | ||
func concat(i1,i2 string) (string, error) { | ||
return i1 + i2, nil | ||
} | ||
``` | ||
|
||
So why use a more complex and verbose approach? Of course, it doesn't make sense to use F-Mesh for simple tasks like concatenating two strings or similar operations. However, when dealing with a component that has tens or even hundreds of input and output ports, that's when F-Mesh truly shines with its declarative syntax! | ||
|
||
|
||
Next, let's add the component to the mesh so it recognizes it | ||
|
||
```go | ||
// Add component to mesh | ||
fm.WithComponents(c) | ||
``` | ||
>[!IMPORTANT] | ||
All components must be explicitly added to the mesh. | ||
|
||
### Passing initial signals | ||
|
||
So far so good, now we can pass some data into the mesh: | ||
|
||
```go | ||
// Pass input signals to mesh | ||
c.InputByName("i1").PutSignals(signal.New("Hello")) | ||
c.InputByName("i2").PutSignals(signal.New(" World!")) | ||
``` | ||
|
||
There is no special API for initialising the mesh, you just put some signals on input ports of any component you want. In real-world scenario you probably will know which components are entry points for the initial data. Putting signals does not trigger anything, it is just like appending an item to slice. | ||
|
||
### Running the mesh | ||
|
||
Now everything is ready to start the computations and run the mesh: | ||
|
||
```go | ||
// Run and check errors | ||
_, err := fm.Run() | ||
if err != nil { | ||
fmt.Println(fmt.Errorf("F-Mesh returned an error: %w", err)) | ||
} | ||
``` | ||
|
||
First, we pass the input signals, and only then do we run the mesh. The mesh is not a long running process by default (though it can be in [more complex use-cases](https://github.com/hovsep/fmesh/blob/main/examples/async_input/main.go)). It behaves more like a computational graph: you build it from components, initialize it with some state (input signals), and then execute it. Running an F-Mesh means triggering activation cycles until a terminal state is reached: an error, a panic, or a natural stop. The "happy path" is the natural stop, which occurs when no component has signals on any input ports. You can learn more about scheduling rules [here](https://github.com/hovsep/fmesh/wiki/Scheduling-rules). | ||
|
||
>[!NOTE] | ||
***Run()*** is a blocking operation — it will block until the mesh reaches a terminal state. As a result of running the mesh, you will receive an activation cycles report and any errors that occurred. The report can be useful for debugging or visualizing the mesh. In the code above, the report is replaced with blank identifiers for simplicity. | ||
|
||
### Getting the results | ||
|
||
```go | ||
// Extract results from mesh | ||
resultPayload := fm.Components().ByName("concat").OutputByName("o1").FirstSignalPayloadOrNil().(string) | ||
fmt.Println("Got from mesh: ", resultPayload) | ||
``` | ||
|
||
There are no surprises here—simply retrieve the signals from the output ports of the respective components (you or the mesh author should know where to place inputs and where to retrieve outputs). And that’s essentially it. If everything is working correctly, you should see something like: | ||
|
||
`Got from mesh: Hello World!` | ||
|
||
By now, we hope you have a sense of how small chunks of data (signals) flow in and out of components. That’s what FBP and F-Mesh are all about: rather than writing imperative code, you describe how your data flows, making it more natural for your task. | ||
|
||
Congratulations! You've just built your first F-Mesh! | ||
|
||
### Chained API and error handling | ||
|
||
The framework utilizes a chained API to minimize repetitive error checks, allowing you to focus on actual programming. You only need to check for errors once at the end of the chain: | ||
|
||
```go | ||
if sig := c.Inputs().ByName("invalid port name").Buffer().First(); sig.HasErr() { | ||
// Handle error | ||
} | ||
``` | ||
|
||
All main APIs have ***HasErr()*** and ***Err()*** methods, which can be used to check if an error occurred in the chain of calls. Errors are propagated up the chain to the parent level (e.g., signal → signal group/collection → port → port group/collection → component → components collection → F-Mesh). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
## Overview | ||
|
||
In F-Mesh, the data exchanged between components is represented as **signals** — flexible units of information that can carry any type of payload. Signals serve as the foundation for communication within the mesh, enabling the dynamic flow of data between components. Each signal encapsulates exactly one piece of data, referred to as its **payload**. | ||
|
||
To explore the full capabilities of signals, refer to the [signal package API](https://pkg.go.dev/github.com/hovsep/fmesh/signal). | ||
|
||
## Payload | ||
|
||
The payload of a signal can be any valid Go data type, offering immense flexibility. This includes primitive types like integers and strings, complex types like structs and maps, or even nil. | ||
|
||
It’s important to note that **every signal in F-Mesh always has a payload**, even if that payload is nil. There is no concept of an "empty" signal, ensuring consistency in data handling across the mesh. | ||
|
||
Example of creating a simple signal: | ||
|
||
```go | ||
mySignal := signal.New("example payload") // A signal with a string payload | ||
``` | ||
|
||
## Signal Groups | ||
|
||
While individual signals are useful for most cases, there are scenarios where working with a **group of signals** simplifies the design. A signal group aggregates multiple signals, each potentially carrying a different type of payload. | ||
|
||
This flexibility allows components to process diverse datasets in a unified manner. For instance, you could group signals with an integer, a **nil**, a slice, and a map payload, and handle them collectively. | ||
|
||
## Creating a Signal Group | ||
|
||
Here’s an example of creating a signal group with mixed payload types: | ||
|
||
```go | ||
mySignals := signal.NewGroup(1, nil, 3, []int{4, 5, 6}, map[byte]byte{7: 8}) // Group of 5 signals | ||
``` | ||
|
||
In this example the group contains five signals with the following payloads: | ||
1. An integer (1) | ||
2. A nil payload | ||
3. Another integer (3) | ||
4. A slice of integers ([]int{4, 5, 6}) | ||
5. A map of byte-to-byte values (map[byte]byte{7: 8}) | ||
|
||
This versatility makes signal groups a powerful tool for managing complex data flows in F-Mesh. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
## Overview | ||
|
||
Ports are the connection points of components in F-Mesh. Unlike systems that follow the actor model (e.g., Akka), where components communicate directly, F-Mesh components are isolated and communicate only through signals passed via ports. | ||
|
||
This design promotes modularity and separation of concerns: a component only interacts with its own ports and does not need to know where its signals are routed. The abstraction ensures that the logic and behavior of each component remain self-contained and decoupled from the overall mesh structure. | ||
|
||
|
||
You can explore the full API for ports [here](https://pkg.go.dev/github.com/hovsep/fmesh/port). | ||
|
||
## Types of Ports | ||
|
||
There are two types of ports in F-Mesh: | ||
|
||
1. **Input Ports**: Receive signals for the component to process. | ||
2. **Output Ports**: Send signals to connected components. | ||
|
||
Both input and output ports are instances of the same underlying type, [Port](https://github.com/hovsep/fmesh/blob/main/port/port.go#L16). | ||
However, F-Mesh enforces type-safe connections: | ||
|
||
* Pipes cannot be created between two input ports or two output ports. | ||
* Connections are strictly from an output port to an input port. | ||
|
||
## Internal Structure | ||
|
||
Each port has three core elements: | ||
|
||
1. **Name**: Unique name used to access port. | ||
2. **Signal Buffer**: Holds an unlimited number of signals. | ||
3. **Pipes List**: Tracks outbound connections to other ports. | ||
|
||
>[!IMPORTANT] | ||
The port buffer does not guarantee any specific order for the signals it holds. | ||
Your program must not rely on the observed order of signals. | ||
|
||
## Working with Groups and Collections | ||
|
||
When working with multiple ports, F-Mesh provides two utility types: Group and Collection. These abstractions simplify interactions with multiple ports, although users typically do not need to interact with them directly. | ||
|
||
## Group | ||
|
||
A [Group](https://github.com/hovsep/fmesh/blob/main/port/group.go#L13) is essentially a wrapper around a slice of ports. It allows you to iterate over and manage a set of ports without concern for their individual names. | ||
|
||
## Collection | ||
|
||
A [Collection](https://github.com/hovsep/fmesh/blob/main/port/collection.go#L15) is an indexed structure, where ports are stored and accessed by name. Unlike a Group, a Collection cannot contain multiple ports with the same name, ensuring a unique mapping of port names to ports. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
A mesh with only one component is rarely useful. The true power of FMesh emerges when multiple components are connected using **pipes**. By connecting components, you enable them to exchange signals and tackle complex problems in small, manageable steps. This idea is not unique to FMesh—it draws inspiration from concepts in object-oriented and functional programming. However, FMesh implements it in a simpler, more transparent way. | ||
|
||
Pipes are intentionally simple: they merely connect two ports. In fact, "pipe" is an abstraction—there is no dedicated "pipe" entity in the source code. Instead, pipes are represented as groups of outbound [ports](https://github.com/hovsep/fmesh/blob/main/port/port.go#L21). Each pipe connects exactly one output port to exactly one input port. While we initially explored more flexible "any-to-any" pipes, this approach introduced unnecessary complexity and was ultimately avoided. | ||
|
||
This design doesn't limit your ability to create ***one-to-many*** or ***many-to-one*** connections. Such configurations are achieved using multiple pipes, each maintaining the simplicity of a single output-to-input link. | ||
|
||
## One to one connection | ||
|
||
|
||
![](https://github.com/user-attachments/assets/c833488a-5d39-4624-8410-f068c10c8d26) | ||
|
||
To demonstrate how pipes work, let’s start with a simple example: | ||
|
||
```go | ||
//Connect output port "o1" of component "c1" to input port "i1" of component "c2" | ||
c1.OutputByName("o1").PipeTo( | ||
c2.InputByName("i1"), | ||
) | ||
``` | ||
**Semantics**: All signals put to the output port ***o1*** of ***c1*** will be transferred to the input port ***i1*** of ***c2***. | ||
|
||
>[!IMPORTANT] | ||
Signals are always copied by reference in FMesh. Instead of creating physical copies, FMesh moves pointers between ports. This means signals are removed from the source port's buffer and appended to the destination port's buffer. | ||
|
||
## One-to-Many Connections | ||
|
||
![](https://github.com/user-attachments/assets/09f18fe8-a3eb-444a-b6e9-08fc7ce658ed) | ||
|
||
To create one to many connection we create multiple pipes from the same source port: | ||
|
||
```go | ||
//Create three pipes from "o1" | ||
c1.OutputByName("o1").PipeTo( | ||
c2.InputByName("i1"), | ||
c3.InputByName("i2"), | ||
c4.InputByName("i3"), | ||
) | ||
``` | ||
|
||
**Semantics**: All signals put on ***o1*** will be copied by reference to all three destination ports (***i1***,***i2*** and ***i3***). | ||
|
||
>[!IMPORTANT] | ||
>Signals are always copied by reference. To avoid unexpected behavior: | ||
>* Keep signals immutable whenever possible. | ||
>* Design components as pure functions to minimize side effects. | ||
>* If necessary, create a deep copy of the signal before modifying it. | ||
Understanding how pointers work in Go is crucial for working with FMesh pipes effectively | ||
|
||
## Many-to-One Connections | ||
|
||
![](https://github.com/user-attachments/assets/2d367069-f267-4d70-9064-c60162500720) | ||
|
||
Creating a **many-to-one** connection is equally straightforward using the same API: | ||
|
||
```go | ||
// Connect multiple source ports to the same destination port | ||
c1.OutputByName("o1").PipeTo(sink.InputByName("i1")) | ||
c2.OutputByName("o1").PipeTo(sink.InputByName("i1")) | ||
c3.OutputByName("o1").PipeTo(sink.InputByName("i1")) | ||
``` | ||
|
||
**Semantics**: This configuration creates three pipes from the output ports of different components (***c1***, ***c2***, and ***c3***) to the same input port (***i1***) of the ***sink*** component. All signals from the respective source ports will appear on ***i1*** | ||
|
||
You can find similar examples in [this integration test](https://github.com/hovsep/fmesh/blob/main/integration_tests/piping/fan_test.go). | ||
|
||
## Cyclic Connections | ||
|
||
![](https://github.com/user-attachments/assets/7bec7fc4-12ec-4583-99ec-611d4a7ad88d) | ||
|
||
In some scenarios, you may need to create a **cyclic connection**, where a component’s output port is connected to one of its own input ports. This is fully supported and a common pattern in FMesh. Just as functional programming favors recursion over loops, cyclic pipes enable components to "self-activate." | ||
|
||
Example: | ||
|
||
```go | ||
// Connect the output port "o1" to the input port "i1" within the same component | ||
c1.OutputByName("o1").PipeTo(c1.InputByName("i1")) | ||
``` | ||
|
||
**Semantics**: The component will reactivate itself in the next cycle, provided there is at least one signal on any of its input ports. This allows the component to control when to activate and how many cycles to execute before stopping | ||
|
||
For a practical example, check out [the Fibonacci example](https://github.com/hovsep/fmesh/blob/main/examples/fibonacci/main.go#L40), which demonstrates how cyclic pipes can be used to implement recursive logic. | ||
|
||
That’s all you need to know about pipes in FMesh. Pipes are the backbone of communication between components, enabling you to build modular, efficient, and highly flexible systems. With the fundamental building blocks of ports and pipes, you can create a wide range of powerful and versatile patterns, including: | ||
|
||
* **Event Bus**: Centralized communication for decoupled components. | ||
* **Fan-In**: Merging multiple inputs into a single unified output. | ||
* **Fan-Out**: Distributing a single input to multiple destinations. | ||
* **Broadcast**: Simultaneously transmitting a signal to all connected receivers. | ||
* **Chain of Responsibility**: Passing signals through a series of components, each capable of handling or forwarding them. | ||
* **Load Balancer**: Distribute incoming signals across multiple components to balance workload. | ||
* **Round-Robin Distributor**: Sequentially route signals to multiple components in a cyclical order. | ||
* **Pipeline**: Process data through a series of components, each performing a distinct transformation or operation. | ||
* **Aggregator**: Collect signals from multiple sources and combine them into a single output. | ||
* **Filter**: Route signals selectively based on specific criteria or conditions. | ||
* **Splitter**: Divide a signal into smaller parts and distribute them to different components for parallel processing. | ||
* **Priority Queue**: Process signals based on their priority levels, ensuring high-priority tasks are handled first. | ||
* **Dead Letter Queue**: Capture signals that cannot be processed by any component for later analysis or retries. | ||
* **Observer Pattern**: Notify multiple components of changes in a shared signal or state. | ||
* **State Machine**: Use cyclic connections to implement state transitions driven by signals. | ||
* **Pub-Sub (Publish-Subscribe)**: Allow components to subscribe to specific topics or signals and receive updates dynamically. | ||
* **Circuit Breaker**: Monitor signals and temporarily halt processing when a failure threshold is reached. | ||
* **Rate Limiter**: Throttle the flow of signals to ensure components are not overwhelmed. | ||
* **Retry Logic**: Automatically reprocess failed signals after a specified delay or condition. | ||
|
||
These are just a few examples. The simplicity and flexibility of FMesh make it possible to design and implement countless other patterns tailored to your specific needs. |
Oops, something went wrong.