Feature suggestion: wl-paste stdout --watch mode

[fedy-cz]

While the wl-paste --watch mode is really useful for continuous clipboard monitoring, I find it kind of awkward that it always requires a command to execute. Sometimes I just need to output the plain text to <stdout> to be processed by another util / script. You can use something like:

wl-paste -t text -w cat

But the additional cat exec on every clipboard change is just unnecessary.

Maybe wl-paste -w - could just use a direct output?

And it would be also nice to be able to specify a separation character between clipboard changes (like a null character or a newline).

[fedy-cz]

One more issue with the workaround described above is that the wl-paste won't monitor for a broken pipe / stdout (why should it, it's not using it) so it never terminates & just keeps running even if the "receiver" is no longer there.

Maybe there could be an option to check the returned error code from the invoked command & terminate if not 0?

[bugaevc]

Hi!

You can use something like:

wl-paste -t text -w cat

Yes, that's what you should do if you just want to concatenate clipboard contents to stdout.

But the additional cat exec on every clipboard change is just unnecessary.

wl-paste always spawns a command to perform the copy. It is implicitly a cat when you don't use the --watch mode — in other words, a plain wl-paste invocation also spawns a cat. With --watch mode, it's explicit, and you can specify something different than cat if you want to.

So it is necessary, unless what you're suggesting is we implement the copying in-process, which is of course possible, but I've intentionally avoided doing that, because hopefully the cat implementation is optimized to use splice(), sendfile(), etc. when supported and beneficial, and I don't want to get into reimplementing that.

And it would be also nice to be able to specify a separation character between clipboard changes (like a null character or a newline).

You could use a cat-like program that does just that. Perhaps a simple shell script like cat && echo.

The idea of the --watch design was not to implement every potentially useful feature into wl-clipboard, but to provide the ultimate flexibility by following the Unix philosophy. There are a myriad of tools that will take input on stdin and send it wherever you want (cat to stdout, write to syslog, append to a file, save into an sqlite DB, pass to some daemon over D-Bus, push somewhere over network, ...), in any format of your choice, and you can plug any of them into wl-paste --watch. Hopefully you don't change clipboard contents often enough that a fork/exec becomes a performance bottleneck.

One more issue with the workaround described above is that the wl-paste won't monitor for a broken pipe / stdout (why should it, it's not using it) so it never terminates & just keeps running even if the "receiver" is no longer there.

Maybe there could be an option to check the returned error code from the invoked command & terminate if not 0?

This is actually a good idea, thanks! If we see that the child was killed by SIGPIPE, maybe we should exit along with it too. Not sure whether we should do anything for other non-0 returns though.

[fedy-cz]

wl-paste always spawns a command to perform the copy. It is implicitly a cat when you don't use the --watch mode — in other words, a plain wl-paste invocation also spawns a cat. With --watch mode, it's explicit, and you can specify something different than cat if you want to.

OK. Maybe I should study up on my Wayland protocols, but why is that? Isn't wl-paste currently the receiver of the clipboard contents? Why invoke another process?

So it is necessary, unless what you're suggesting is we implement the copying in-process, which is of course possible, but I've intentionally avoided doing that, because hopefully the cat implementation is optimized to use splice(), sendfile(), etc. when supported and beneficial, and I don't want to get into reimplementing that.

OK. I should probably read up on this first (or just look at the code). But how does this work exactly? Does wl-paste just get something like a file descriptor from Wayland and it gets passed to another process (wl-paste never touching the actual data)? Is performance really such a concern? My typical (textual) clipboard content is usually a few kB at best, so a simple copy using something like a 4K buffer should be completely fine. I would suspect that any performance gains that you could see from "efficient" implementation of cat will be eaten away by the exec call in 99.9% of cases.

You could use a cat-like program that does just that. Perhaps a simple shell script like cat && echo.

I build a special util (a few lines of C code) for that.

This is actually a good idea, thanks! If we see that the child was killed by SIGPIPE, maybe we should exit along with it too. Not sure whether we should do anything for other non-0 returns though.

This would help to solve the most pressing issue. The exec is truly not that bad. I would argue that SIGTERM/SIGKILL might be considered too (because they are intentional). Or let the user specify the exitcodes?

Then again, there is something to be said for the ease of use in the streaming scenario (which in my opinion is the more UNIXy way of doing things).

[fedy-cz]

wl-paste always spawns a command to perform the copy. It is implicitly a cat when you don't use the --watch mode — in other words, a plain wl-paste invocation also spawns a cat.

Also: This should probably be documented? Clipboard contents could be sensitive and if you don't fully control the PATH bad things could happen.

[bugaevc]

OK. Maybe I should study up on my Wayland protocols, but why is that? Isn't wl-paste currently the receiver of the clipboard contents? Why invoke another process?

OK. I should probably read up on this first (or just look at the code). But how does this work exactly? Does wl-paste just get something like a file descriptor from Wayland and it gets passed to another process (wl-paste never touching the actual data)?

Somewhat like that, yeah. Except that it's wl-paste that picks a file descriptor and sends that to the compositor (and the compositor likely forwards it to the other client who owns the clipboard selection), and the other process is expected to write the clipboard contents into this fd.

You'd think that wl-paste could just send its stdout and the other process would write there directly without any additional copying; wouldn't that be cool? Unfortunately there's no indication that the other process is done writing other than the fd getting closed; and wl-paste has to know when data transfer is complete (to print the newline, and to only exit once all the data is written). Hence, it creates a pipe and sends the writing end over Wayland, and spawns a cat (or a user-specified command in --watch mode) to read from the reading end of the pipe. Once the writer closes its fd, the command will properly receive an EOF, and should exit, and then wl-paste will know it's done.

My typical (textual) clipboard content is usually a few kB at best, so a simple copy using something like a 4K buffer should be completely fine. I would suspect that any performance gains that you could see from "efficient" implementation of cat will be eaten away by the exec call in 99.9% of cases.

Yes, for the typical case of a small selection the fork/exec overhead is larger than if we'd just performed the copy ourselves. But hopefully in that case it's still fast enough; at least nobody so far has shown up complaining that wl-paste is actually slow because of the cat. On the other hand, you could copy a multiple-gigabyte image into your clipboard, and it would be unfortunate if we spend minutes shoveling the bytes around when we could do a virtual copy with sendfile() or something.

But also this is not set in stone, I just haven't heard a compelling argument yet.

Also: This should probably be documented? Clipboard contents could be sensitive and if you don't fully control the PATH bad things could happen.

The cat is not the only thing that wl-clipboard spawns either; we use subprocesses somewhat liberally, somewhat like a shell script would. If PATH is set to a malicious value, bad thing may happen, and this is true of any program, not just wl-clipboard.

[fedy-cz]

If PATH is set to a malicious value, bad thing may happen, and this is true of any program, not just wl-clipboard.

You can hard-code the full binary path to avoid that. My opinion is that invoking other binaries should be documented (when not fully expected). It's a dependency / interface with the rest of the system after all ( + the possible security concerns ). And it doesn't need to be malicious to get unexpected behavior.

[bugaevc]

You can hard-code the full binary path to avoid that.

I'm sure the NixOS people will come after me if I do that 😄

Have you ever seen a shell script invoke cat as /bin/cat (or is it /usr/bin/cat?), just in case PATH is set improperly? Me neither.

[fedy-cz]

One more question about serialization:

When the clipboard contents changes in the middle of the invoked command processing it, it's probably fine because the descriptor is still valid (I hope). But if I want the "streaming mode" to be perfectly correct I need to make sure that one is finished writing before the other one starts to.
Does the --watch mode assure that (waiting for one command to finish before the other one is executed) or do I need to implement some kind of locking myself?

[bugaevc]

When the clipboard contents changes in the middle of the invoked command processing it, it's probably fine because the descriptor is still valid (I hope).

As far as Wayland is concerned, yes, the change (replacing an active selection with another one) is atomic. But I wouldn't count on the other client (who used to own the selection and is sending the data to the pipe) to not abort the transfer once the selection is replaced.

Does the --watch mode assure that (waiting for one command to finish before the other one is executed) or do I need to implement some kind of locking myself?

Currently, yes, it runs one copy of the command at a time. I don't think of this as of a hard guarantee though (it's just the behavior that makes sense), so if your storage backend has a convenient & cheap-when-not-contended locking mechanism (e.g. inserting a row into a sqlite DB), it'd be a good idea to make use of it.

[Vaisakhkm2625]

You can hard-code the full binary path to avoid that.

I'm sure the NixOS people will come after me if I do that 😄

As a NixOS user, I sure will be on your doorstep next day... XD