idea: synthetic file system mounted on /flux using 9P

[garlick]

Old idea actually, but having recently revisited diod to do some maintenance, I'm reminded again of how simple it is to embed a 9P server in other code. We could embed one in the flux broker (as a broker module) to get ourselves a synthetic file system mounted on /flux for each Flux instance.

One thought is that a file based interface might be desirable from an application or tool point of view as a way to talk to flux, since it wouldn't require linkage with external libraries. Users could, for example:

• submit jobs by writing JSON to /flux/submit
• list jobs with ls /flux/jobs and access raw job data from /flux/jobs/ID/*
• point HWLOC_XMLFILE at /flux/jobs/self/hwloc.xml

Any number of clever synthetic file system tricks could be possible.

It would be neat if flux proxy could also set up this mount.

A downside is that CAP_SYS_ADMIN is required for the mount(2) system call, so IMP assistance may be required. In addition, the signing mechs from flux-security would probably be needed to implement authentication (which is easily done in 9P) . A lack of universal availability when flux-security is not installed/configured would diminish the value of this as a primary interface. Although maybe it is less hard these days with so much container support everywhere.

Any thoughts on the value of this?

[garlick]

Just realized that for a user instance, if 9P was mounted from a unix domain socket in the rundir which is private, and mounted within a private namespace, 9P authentication might not be required. Just some way to execute the mount system call wtih sufficient privilege.

[wihobbs]

@grondo and I chatted this morning about the possibility of leveraging fuse to help with some of this work. We found some docs about a port of 9P to fuse that could be helpful.

Initially, when I read this, I thought maybe we could leverage an fuse.ext4 filesystem in a mount namespace to get around needing the IMP, since that is deploy-able without CAP_SYS_ADMIN, but I don't think that would work for an idea that Mark succinctly described as:

No the idea Jim posted is a synthetic filesystem, i.e. data directly from the broker is represented in the filesystem as files. Kind of like /proc for the broker

With that in mind, fuse.ext4 probably isn't the right idea, but could a mount namespace still be leveraged to get around needing privilege to mount the 9P file system?

One thought is that the mount namespace probably wouldn't work for a multi-user instance, but could work for a single-user instance.