API implementation test with Qt4

[IvarsKarpics]

Hi all,

I created a mini brick to test the sample changer interface:

In the header of file I have:

from api import sample_changer

api is a directory containing file sample_changer.py:

from HardwareRepository import HardwareRepository

hw_repository_client = HardwareRepository.HardwareRepository()
bl_setup_hwobj = hw_repository_client.getHardwareObject("beamline-setup")

def mount_sample(location, device_name=None, wait=False):

    Mounts sample from location
    Some sample delivery devices like yets could not have a difined locations,
    so location argument could be None.

    :param LocationStr location: location
    :returns: True if mount successful otherwise False
    :rtype: bool

    if not device_name:
        device = bl_setup_hwobj.sample_changer_hwobj
    else:
        device = getattr(bl_setup_hwobj, device_name + "_hwobj")

    device.load_sample(holder_length=None, sample_location=location)

def unmount_current_sample(device_name=None, location=None):

    Un-mounts mounted sample to location, un mounts the sample
    to where it was last mounted from if nothing is passed

    :param LocationStr location: location
    :returns: True if un-mount successful otherwise False
    :rtype: bool

    pass

We could use beamline-setup as container of all hardware objects. By default mount_sample method looks for sample_changer_hwobj and call related method. I added argument device_name because it may happen that several sample mount devices are available.

What do you think about this?

[marcus-oscarsson]

Looks good, device_name is probably a good idea !. Do we really need the wait argument, if we really on a signaling mechanism to handle the result or errors for that matter (as long as they are not handled by exceptions and in that case the calling code needs to handle them)

[IvarsKarpics]

I would suggest to keep the wait as an optional argument, because at some point we would use api to communicate between devices. One other question: how such a api construct could be used for both Qt and web versions? Could you please create a similar test in the web version?

[IvarsKarpics]

Idealistically at some point we could reach the directory structure like :

+ mxcube
|-- + api
     |-- sample_changer.py
     |-- ...
|-- HardwareRepository
|-- qt_gui
|-- web_gui

[marcus-oscarsson]

As I mentioned in the meeting I don't have alot of time to provide an example for the web version right now, Ill try to do my best but I'm not promising anything. The structure of the API is quite similar to what we already use in MXCuBE3. Each UI just needs a layer that routes the incoming calls, HTTP requests, RPC or just function calls to the right API call.

The example you have provided could be used in both WEB-UI and QT providing that the right routing mechanism exists.

I'm not sure that understand what you mean by communication between devices ?, the API is only meant to be used by the user interface.

[rhfogh]

@marcus-oscarsson
I am not sure if the API will only be used by the UI. Surely other components will call the same functions where applicable, rather than haveing an alternative API? The queue is an obvoius example where almost necessary functionality seems to be in the API, so that very few additions makes the UI API into a complete queue API?

[marcus-oscarsson]

Hm, The idea is to provide an API that is to be used exclusively between the UI and the backend. As such I think its convenient if its a non blocking API. The queue and other components should have well defined APIs that are used internally, or between components within the "core". It would in my opinion be a bit strange to route those internal calls through a layer above the core ?

The API could theoretically be extended to include other types of clients, apart from UI's, if that is what you refer to. Such a client could be a workflow like application, but that I think that should be discussed after we have settled the initial API, since it would be extending the scope a bit.

[marcus-oscarsson]

As you mentioned, the "internal" API of some components will be very similar to the one in the UI-API. Queue, Lims, Processing, are probably such components. Queue is a particularly good example because few of its functions are actually used exclusively internally. Its possible that we at some stage can provide a direct mapping to those components if it completely satisfies the needs of the UI-API.

I think its to early to do that today because the current "internal" queue API is not ready for that. Taking time to reflect on what we need from a UI perspective are probably going to give us a better idea if its a good idea to provide such a mapping or not.

[IvarsKarpics]

I created branch called api_test in the mxcube: https://github.com/mxcube/mxcube/tree/api_test
Use api_test.yml gui file to test the brick.

[marcus-oscarsson]

Thanks !

[marcus-oscarsson]

I think that each UI or eventually other client that uses the API needs a layer which "bridges" the different technologies / architectures used. The Qt4 UI can simply use the API as it is within the same process as the rest of the MXCuBE core. However, the WEB app and other REST, or other client-server based, solutions needs something that handles the parts that are specific to the technology used. In lack of a better term I would call this a communication layer.

The implementation of this layer would be fairly straight forward, for direct function calls, like in Qt4, it would be something in the lines of what @IvarsKarpics have provided above. For MXCuBE 3 or anything using REST one would need to do something like this:

from core import ui_api

@mxcube.route("/mxcube/api/v0.1/sample_changer/mount", methods=["POST"])
def mount():
    try:
        location = request.get_json()
        ui_api.sample_changer.mount(location)
    except Exception:
        return Response(status=409)
    else:
        return Response(status=200)

The REST implementation could of course be reused for any application using the REST protocol. This transport layer can be seen as belonging to the UI part of the application while the UI-API would belong more to the server.

[rhfogh]

@marcus-oscarsson
Heartily approve!

I guess the questions are 1) if the Qt4 implementation needs a separate communication layer or can call the API layer directly. 2) if we accept that the API layer functions can have additional parameters that are not part of the official API nor supported by the MXCuBE3 communication layer (e.g. 'wait=False').

[marcus-oscarsson]

I see,

• 1 I guess calling the UI API directly is a kind of implementation, that almost comes for free.

• 2 I would say that its very strange if we define an API for the UI and then change it so that allows us to do other "unspecified things". We could of course say that its acceptable and that its a implementation detail, although it seems to me that it somehow defeats the purpose of the UI-API.

[IvarsKarpics]

Thanks @marcus-oscarsson for the mx3 example. This is exactly what I was looking for. For me calling methods is straight forward (as you said comes for free). Other question is how to handle signals coming from hardware objects. Should we define sample_changer as a hwobj and reemit signals or there is a better suggestion how to pass data to ui layer?

[marcus-oscarsson]

Hm, well I think that the same "design" that I described above can be used for signals as well. The signal/event system is dependent on the protocols used, for instance websokets for REST HTTP services and Qt Signals for Qt.

So one way would simply be to define which handlers that are needed, like we have done in the proposals, and make sure that they are forwarded to the right place. That would mean that the PyDispatcher signals from the Hardware Objects needs to be "relayed" using the protocol specific event mechanism. See example below

import core
from core import ui_api

# Possible dispatcher mechanism, I assume that the sample_changer hardware object is
# accessible directly on the core object
core.sample_changer.connect('loadedSampleChanged', 
    ui_api.sample_changer.signals.loaded_sample_changed)

# MXCuBE3 Example
# Handler defined in UI API sample_changer implementation
def loaded_sample_changed(sample):
    socketio.emit("loaded_sample_changed", sample, namespace="/hwr")

# Qt Example
# Handler defined in UI API sample_changer implementation 
def loaded_sample_changed(sample):
    QObject.emit(core.sample_changer, PYSIGNAL('loaded_sample_changed), (sample,))

My Qt syntax is a bit rusty but I think it should be something in this direction ...

[IvarsKarpics]

Hmm, not sure if this is a best solution because then we strictly define two protocols (socketio and Qt signals). Whats speaks again using the same PyDispatcher signals and use api as a "hardware object"?
Maybe we need a referendum to decide this :)

[rhfogh]

Without having looked at htis in detail:
If we potentially have multiple signal mechanisms, can we call a 'send_signal' function, and modify it according to teh UI we are using?

[marcus-oscarsson]

We can of course provide a wrapper function like Rasmus suggests, called send_signal or emit that calls the right function depending on "transport".