Question on specifying localhost and inter-computer topics

[chengguizi]

May I know If there is a good way to specify only a subset of topics to be available for inter-computer discovery & transimission, while in the network-mode of eCAL?

The motivation is there are many topics created and exchanged within the same computer, mostly for pipelines of data processing. But they are not interesting to other computers (nodes) in the networking. Only a small number of final topics are of interest to be exchanged between computers over network (wired or wireless).

Another motivation is to have natural topic namespace. If we make all topics discoverable for all machines under the same network, we have to manually make sure all nodes are not publishing duplicated topic names. But if we know topics are default only discover-able localhost, then we can only worry about scoping topic names for those topics selected for inter-computer exchange.

[FlorianReimold]

Short answer: No, there is no way to control the network-availability of topics an a per-topic level.

Long answer: All eCAL Processes that run in network mode advertise all of their topics in the network. This behavior is controlled by the ecal.ini.

You can make each process load its own ecal.ini and thus turn network mode on for some processes and off for others. This method however control the network setting for the entire process and not for a certain topic. A process running in local mode will also not only omit advertising its own topics, it will also refuse to subscribe to topics from different machines.

Another workaround is to control the transport layer from your code. In the eCAL CPublisher API you can override the layer that is being used for data transport and just turn UDP and TCP off for topics that are supposed to stay locally. The process will however still advertise the topic in the network and remote processes will be able to subscribe to them - but they will not receive any data.
If you ask me, this solution is unintuitive, error-prone and not very generic.

My suggested solution would be to prefix your topics with your hostname. So you don't publish them in a plain list like topic1, topic2, ... any more, but group them by hostname, like HOST_A/topic1, HOST_A/topic2, HOST_B/topic1. While this will not prevent you from subscribing to the data from any remote process, you can re-use the same "plain" topic names on every machine (because now they are prefixed) and always see and control where your data is coming from.

Another workaround is to not use the eCAL network mode at all, but run a proxy application that subscribes to a specific set of topics on one PC (via SHM), transports those over network and re-publishes them via SHM.
We have an application (ecal_to_tcp and tcp_to_ecal) available for that in the tcp_pubsub repository. As the name suggests, it uses TCP to transfer the data over network. It can be enabled by setting the option TCP_PUBSUB_BUILD_ECAL_SAMPLES=ON during CMake.

---

@hannemn
Interestingly we have already thought about making this an eCAL Feature, as the requirement comes up from time to time.

[chengguizi]

An eCAL feature on this would be really handy, like a "namespace" feature. The suggested solution would work, but need to make sure everyone development the system has the same agreement on naming, and adding the correct prefix for every program they wrote.