Clipboard suggestions

[codokie]

I made the primary clipboard appear in the suggestion strip view as a suggested word whenever neutral suggestions are to be shown like in a beginning of sentence (fixes #167).
Here is an example of the feature:

In android 7.0+, the API allows detection of sensitive clipboard content, and when that happens the content is redacted like this:

Suggestions will disappear if they are picked or 3 minutes have passed.
There is also new setting in Suggestions settings that allows to toggle the suggestions.
There will be a suggestion for number input fields only if the clipboard content is a number.

P.S. You can long press the clipboard suggestion and then press the X icon to dismiss it

[RHJihan]

Nice work. ❤
One question: Is there a time limit for how long old clipboard content will be suggested?
The clipboard icon from the toolbar can be used for consistency, perhaps?

[Helium314]

As usual, the PR is already 2 days old but I didn't even have time to go through the code...

I haven't tested, but "whenever neutral suggestions are to be shown" might not be quite the right thing.
HeliBoard might start showing suggestions right away when opening an input field, but when this is right after copying, the clipboard suggestion might still be wanted. Though the clipboard suggestion could be added to the list of normal suggestions in such a case.
On the other hand seeing the clipboard suggestion often for a long time might be seen as annoying / unnecessary.
I'm thinking about when it should not be shown any more. I'll ask in the issue to reach a few more users.

And maybe the icon should be the normal clipboard icon instead? I can imagine the icon looking a bit out of place depending on the theme / colors.

Maybe also the input field type should be considered, because e.g. clipboard text content should not be shown for a number field.

[RHJihan]

I'm thinking about when it should not be shown any more. I'll ask in the issue to reach a few more users.

My opinion is up to 90 seconds right after copying any text. Samsung keyboard does this for about 120 seconds.

[RHJihan]

Suggestions will disappear if they are picked or if they are older than max history retention time.

I am a little bit confused. What if someone toggles off the clipboard history (the default is 10 min) or sets the retention time to a bigger one? Or set that to No limit? Is it convenient to match the history retention time or to set some specific time, i.e., 90 seconds or 120 seconds?

There will be a suggestion for number input fields only if the clipboard content is a number.

It's impressive. Loved it.

The unicode clipboard icon has been replaced with the default clipboard icon.

It's looking good.

[RHJihan]

I can easily set it to a constant time but personally feel that even 120 seconds is too short. If I copied something and something has caught my attention for 2 minutes (may happen frequently), I still want to be suggested the copied content afterwards.

I will suggest seeing what other keyboards are doing in this situation, i.e., Gboard, Swiftkey, or Samsung Keyboard. I guess it will bring a better user experience.

[RHJihan]

Is the time limit specified somewhere? I'm not 100% sure but In Gboard at least the suggestion doesn't seem to disappear after 120 seconds

on my phone's Gboard the clipboard suggestion disappears after about 120 seconds. Is this more than 120 seconds on your phone?

[RHJihan]

Actually it is inconsistent, should I revert it back?

I am confused here, too. But I think clipboard suggestions shouldn't be very long. Anyone needing clipboard content after a specific time can always have it on the clipboard history view.

I tested the PR. Here are my thoughts:
If the history retention time is less than 120 seconds, I think clipboard suggestions should match that time.

If I copy text when another keyboard is active, after 120 seconds, it still shows as a clipboard suggestion on this PR. I am not sure if it is related to this PR. Maybe it's start counting the time after loading the keyboard. But it's not an important thing after all.

I am not that level of coder. So, I am giving my experience and perspective as a user and/or tester.

I think it would make more sense to show the main toolbar instead of the "neutral" suggestions. That's how it works in GBoard etc.

I think it's cool.

[RHJihan]

It is perfect now for me.

To fix this I can move those variables to the companion object.

After a quick try, this still happening. Force stop the keyboard app and enable it again will suggest old clipboard content as suggestion.

[RHJihan]

long press the clipboard suggestion and then press the bin icon to remove it

Is Samsung Keyboard's way of deleting suggestions better?

[RHJihan]

By the way, I would appreciate some help in making the suggestion background distinct from the suggestion strip view background like in other apps. I tried modifying it programmatically but it seems to affect only the pressed state of the TextView.

As I said earlier, I am not that level of coder. So I cannot help you with this. @Helium314 might help you.

I could adjust the bin button to just dismiss the suggestion without affecting the primary/internal clipboard, but I think the icon may convey a message that the entry should be deleted.

What about both? X button to dismiss the clipboard suggestion, but it can be accessed later on the clipboard history if a user would like to retain history. And the long press bin button to delete the clipboard entry.

I'm probably asking for much change. I am sorry for that.

[RHJihan]

On a second thought, such a button would have to replace the pinned toolbar keys to ensure the suggestion strip view is not cluttered/confusing.

Is it possible to place that button after any pinned toolbar button or on the left side right after the toolbar button?

[RHJihan]

If you think having 2 keys is redundant, I agree with that. I like your simple approach.

For dismissing the clipboard suggestion, the x button is more discoverable compared to the long press dismissal from my point of view. What do you think about this?

[RHJihan]

I am totally delighted with the way it is now, and I am waiting for this PR to be merged.

Maybe it should be added to the "Hidden Functionality" readme?

Yes, you can mention this to the Readme and Hidden features.

[Helium314]

I wonder, is there a way to put the clipboard suggestion like an inline suggestion view?
It seems more suitable to me, as the clipboard suggestion is similar to inline autofill suggestion as in being a differently styled suggestion that is shown alone (without any other suggestions).

[Helium314]

I have to agree with you that the inline suggestions styling seems like a good fit for the clipboard suggestions, but I don't know yet if we can just 'steal' the style.

Right, I hadn't noticed that the style is actually coupled to inline suggestions.

Code-wise the inline suggestions use a specialized API to communicate with the autofill app, it may seem out of place to whoever reads the code if we use some of it for a different purpose (if that's even possible).

It should be easy to generalize the inline suggestion part in SuggestionStripView, after all it's just setting a View and keeping track of the visibility.
My idea was to use setInlineSuggestionsView to set some customized clipboard view like

    public View createClipView(String clipText) {
        TextView view = new TextView(getContext());
        view.setOnClickListener((v) -> mListener.onClipboardSuggestionPicked(clipText));
        // add some styling
        return view;
    }

and onClipboardSuggestionPicked calls LatinIME.onTextInput.
The view also has a handler, so you could e.g. trigger removal after a certain time using view.postDelayed.

[Helium314]

Since you still don't use setInlineSuggestionsView and didn't provide information on why, I had a deeper look, and I think it should be possible (and preferable, as per contribution guidelines).

The idea when clipboard suggestion could be shown:
First LatinIME checks whether clipboard suggestions are wanted, and if yes calls something like mClipboardHistoryManager.createClipboardSuggestionView(editorInfo) (analogous to InlineAutofillUtils.createView).
If there is suitable clipboard content, ClipboardHistoryManager returns a non-null view with the listeners already in place, and LatinIME calls mSuggestionStripView.setInlineSuggestionsView (as mentioned previously, this could be renamed).

Is there anything fundamentally wrong with this approach?

[Helium314]

it would litter ClipboardHistoryManager,

If you think ClipboardHistoryManager is going to grow too much, you could have the view creation happen in a utility file.

The re-use of the inline suggestion view should also reduce the need for clipboard-suggestion specific code in LatinIME.

Also creating a new textview each time would be less efficient than just keeping reference to it since only the text needs to be altered

Possibly, depending on how often it's really created. If this is an issue, a prepared view can be cached, and updated before use.

but there would be a need for an indication that the view is indeed a clipboard suggestion

Why is this? I don't have a full overview over the PR right now.
In any case it should not be an issue, the tag can be set to "clipboard" or something like that.

[Helium314]

Seriously, those parts of the contribution guidelines have been there since OpenBoard times. And they don't exist just because someone likes writing guidelines.
For me the whole process is also very frustrating, as I have to discuss in order to convince you to follow these guidelines, and not just in this PR.

I really would like to spend my time on code instead of discussions, but this kind of adding yet another thing for almost the same functionality is making the app harder to maintain. And I understand you might not be concerned about maintainability ("won't affect the appearance or usability of the feature").
But I need to maintain this app, and in some places it's already a nightmare where changes are very prone to have unintendend effects. I'm not exactly innocent in this respect (specifically looking at the keyboard parsing, which I tried to improve several times already for this reason), but there is a difference between this sort of issue arising for historic reasons and when it's plainly in sight and can be avoided with relatively low effort.
Re-using existing mechanism consistently contributes to maintainability of the code, because when making a change (much) later I don't have to figure out / remember why there are two different ways of showing a single "special" suggestion, and when doing changes I only have to be careful about not breaking one functionality instead of two.

I'm trying to improve the current situation, and I've had "Internal cleanup (a lot of over-complicated and convoluted code)" in the to do section for a while now. Currently it feels like I have to not just (slowly) work on this, but also have to do discussions over and over again that such things do not find their way into the app via PRs.

I have "still it would be good if you announced that you are working on it, so we can discuss how changes are best implemented" in the contribution guidelines, and I think discussing on how to implement larger changes before most of the code is written is a good way to avoid me asking to completely change your code.
Could be in the issue, or by starting a draft PR of a (possibly) non-functional version sketching your idea.

I'm sorry if this comment comes across as overly criticising, so I want to explicitly mention that I definitely do value your contributions (and I'm not happy that I don't have time to keep up with your pace of PRs).

[Helium314]

I adjusted the PR to use the (now renamed) setInlineSuggestionsView, and create the view in ClipboardHistoryManager.
Behavior is same, or very close to, except for:

• always showing the dismiss icon (I assume users may want to dismiss the suggestion relatively often and should have an easy way to do it)
• suggestion is not disappearing automatically after 3 min (I think there is no real benefit from this)

Further I removed some crash sources (using methods / resources not available on devices with the current minApi).

I'll have another look at it in the next few days and plan to merge it then.

[RHJihan]

I haven't tested this merged PR yet. But I just wanted to know:

suggestion is not disappearing automatically after 3 min (I think there is no real benefit from this)

So, will the clipboard suggestion appear every time (if the clipboard has something copied already) I try to type something unless I dismiss it manually?

[Helium314]

It will not appear if the clip is more than 3 min old. But it will not disappear on its own if you just wait and stare at the keyboard for 3 minutes.

[Helium314]

I hope the implementation is ok for you. I tried to keep behavior close to your version when moving most of the stuff into that one view.
(Will also work on your other PRs, but that will take some time. Especially because I'll be barely available for the next 3 weeks.)