Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Any way to import this for iOS? #4

Open
devanshuDesai opened this issue Jul 28, 2020 · 9 comments · May be fixed by #91
Open

Any way to import this for iOS? #4

devanshuDesai opened this issue Jul 28, 2020 · 9 comments · May be fixed by #91

Comments

@devanshuDesai
Copy link

The iOS GBoard app doesn't seem to support modifying the personal dictionary.

@DenverCoder1
Copy link
Owner

iOS Gboard does not currently have a way to import shortcuts. You can add shortcuts 1 at a time on iOS using Text Replacements in the iOS settings, although there doesn't seem to be a way to import a file. Shortcuts added in the iOS settings should also work in Gboard.

It's possible there is another keyboard app for iOS which allows importing shortcuts and a file could be created for it, but I don't know which might and I don't have a way to test it right now.

@hhliu0
Copy link

hhliu0 commented Sep 13, 2023

It looks like this might be possible (since at least macOS Catalina 10.15) by first importing a list of Text Replacements on Mac, then syncing to iOS via iCloud. https://support.apple.com/guide/mac-help/back-up-and-share-text-replacements-on-mac-mchl2a7bd795/13.0/mac/13.0
Tool that could probably be adapted to do the conversion: https://github.com/KyleBing/wubi-dict-editor (via https://forum.keyboardmaestro.com/t/is-there-an-automated-way-to-import-apple-keyboard-text-replacement-shortcuts-into-km/19648/3)
Of course at that point, it wouldn’t be a GBoard dictionary, so if this turns out to work, should we fork this repo? Or expand it to work with macOS/iOS, to keep new symbols added in sync?

@DenverCoder1
Copy link
Owner

DenverCoder1 commented Sep 14, 2023

Thanks for finding this!

It only took a few minutes to generate some python code to convert the dictionary to a plist file using the built-in plistlib library.

Here's the output:

https://raw.githubusercontent.com/DenverCoder1/latex-gboard-dictionary/88c9f2cdb89798819610b97a8a708ed6be2bd2fb/dictionary.plist

I have not tested it, but if someone would like to and let me know if it works, that would be helpful.

Apparently, you can just drag and drop the .plist file to the  → System Preferences… → Keyboard → Text pane to import it.
https://forum.keyboardmaestro.com/t/is-there-an-automated-way-to-import-apple-keyboard-text-replacement-shortcuts-into-km/19648/12

@DenverCoder1
Copy link
Owner

I think keeping it the same repo is fine. It would make it easier to keep it updated since we can automate generating the plist file from the dictionary.txt on every push.

We'll just need to make sure it works and add some documentation on how to use the file.

@DenverCoder1
Copy link
Owner

After some searching, I ended up finding that it seems that someone else has already made a version of LaTeX shortcuts for Mac / iOS, so maybe it's not needed for me to add it to this repo.

https://github.com/tslmy/latex-input-iOS-macOS

@hhliu0
Copy link

hhliu0 commented Sep 14, 2023

Wow, thanks for working on this so quickly! And commendable Google-fu, I couldn't manage to find that repo myself (granted, I got too excited and stopped looking in order to comment here as soon as I found out importing was possible... lesson learned). That being said, it looks like that repo hasn't been updated since 2017, not to mention it's significantly less popular than this one (meaning more people in my position would find this), so I personally think it would be a good idea to keep an up-to-date plist here too. Perhaps just mention/credit that other one? Especially if we merge any new commands from that into this, I saw it had almost 1000 as well.

@hhliu0
Copy link

hhliu0 commented Sep 14, 2023

Ok, I just tested importing the plist file on my Mac. It threw an error (see below) and seemed to crash the Text Replacements floating pane in Settings.

image

But upon reopening (I think just the pane, not Settings edit: the actual Settings app didn't crash, but all the panes did, as in selecting any section in the sidebar, it would be blank, so I did have to restart Settings), all the shortcuts seemed to import correctly, but some characters (e.g. ^inf) show up as square with ?, I'm guessing because macOS's fonts don't yet support those Unicode code points.

image

But the main thing I wanted to make sure was that it doesn't overwrite existing shortcuts (Text Replacements), which I can confirm it doesn't. Of course, I backed up my existing ones, and fwiw the header on that generated backup plist is identical to yours, so that's not the issue causing the error message. Maybe it's just a display issue with 1000+ new lines being added to that pane? The other repo mentions: "For a while, no change may be observed in this listbox. That's because this Text Replacement functionality wasn't really designed to handle definitions as many as such."

If it's just a one-time thing, I guess we needn't worry too much, but it would still be nice to fix if possible, assuming it's not a bug in macOS itself which it sounds like it might be: https://discussions.apple.com/thread/254921358

Update: The replacements list successfully synced automatically to iOS via iCloud, probably within a couple minutes. I also found in testing that every time you make a change to the list, it takes anywhere from a few seconds up to a minute or so to reload everything, during which time no shortcuts work. The interface is can also be a bit slow to respond while trying to edit, again probably because it wasn't designed to handle this many definitions.

Anyway, the shortcuts are working on my phone (albeit still missing the few ones like ^inf), and that was my main goal in the first place, so I'm happy about that!

image

HOWEVER, see my next comment about some caveats, including a very major one...

@hhliu0
Copy link

hhliu0 commented Sep 14, 2023

Okay so I've spent a bit of time doing some fairly extensive testing, and it's mostly working, with the following caveats (trying to be thorough in documenting everything I've tried so far):

  1. The biggest issue by far is related to one mentioned at https://github.com/tslmy/latex-input-iOS-macOS: "macOS seems to have its own logic handling upper cases in text substitution, which cause \alpha and \Alpha to both yield Α." It turns out to be much, much more complicated and confusing than that, and has honestly been doing my head in a bit...

At first, I found that it does actually treat \alpha and \Alpha as different sequences for me, correctly (and consistently) giving α and Α, on both macOS and iOS. So I thought, that's good, they must've made a change under the hood sometime after 2017, maybe everything just works now! However, when I tried typing \mathfrakl and \mathscrl in a test message ℍϵ𝔏𝐿☉, I got \mathfrakL and \mathscrL instead.

I thought maybe it preferred the capital version because that one came first in whatever lexicographic order the shortcuts are processed in, where uppercase comes before lowercase (it's actually the other way for how they're displayed in the replacements list, but that turns out to be a moot point). Nope: \barm and \barM both produce m̅. I then thought, surely it's the order they're listed in and thus imported from the plist then? Sure enough, \barm came before \barM while \mathfrakL came before \mathfrakl. (This hypothesis turns out to be incorrect too, as we'll see.)

I thought ok, if only recognizes when the capital is the first letter in the sequence (based on \alpha and \Alpha), we could use that to for example redefine \mathfrakL and \barM as \Mathfrakl and \Barm Unfortunately not so: \Beta and \beta both gave Β, immediately dashing my hopes.

At this point, I realized I needed to do some systematic testing. I typed out each of the following series of letters (manually, because copy-pasting a prefix like \mathfrak and typing the remaining letter doesn't trigger a replacement), repeating a few times to check consistency (i.e. that it's not choosing upper vs lower randomly), and got these outputs:

\Alpha to \Omega: Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο Π Ρ Σ Τ Υ Φ Χ Ψ Ω -- Good, as expected because Greek capitals all come before Greek lowercase

\alpha to \omega: α Β Γ Δ ϵ ε ζ Η Θ ι Κ Λ μ Ν ξ ο π Ρ Σ ς τ υ Φ φ χ Ψ ω -- WTF?? I fail to see any rhyme or reason to this, it's not the letter length, for example, or even by common usage (hardcoded by some nerdy Apple engineer, idk I was clutching at straws), because lowercase \theta and \lambda are more common by far

\barA to \barZ AND \bara to \barz both: A̅ B̅ C̅ D̅ E̅ F̅ G̅ H̅ I̅ J̅ k̅ l̅ m̅ n̅ o̅ p̅ q̅ r̅ s̅ t̅ u̅ v̅ w̅ x̅ y̅ z̅ -- Started losing it at this point. I double checked that ALL the lowercase \bar come before the uppercase \bar in the plist (and they go \bara \barA \barb \barB etc. in the replacements list pane in macOS/iOS Settings), so we can basically throw out both those theories

\mathfrakA to \mathfrakZ AND \mathfaka to \mathfrakz both: 𝔄 𝔟 ℭ 𝔡 𝔈 𝔉 𝔤 ℌ ℑ 𝔧 𝔎 𝔏 𝔐 𝔑 𝔬 𝔓 𝔔 𝔯 𝔖 𝔗 𝔲 𝔙 𝔚 𝔛 𝔜 𝔷 -- Similar random distribution as Greek lower, obviously not correlated tho

\Bara to \Barz: A̅ B̅ C̅ D̅ E̅ F̅ G̅ H̅ I̅ J̅ K̅ L̅ M̅ N̅ O̅ P̅ Q̅ R̅ S̅ T̅ U̅ V̅ W̅ X̅ Y̅ Z̅ -- Hmm a potentially promising workaround? To be clear, I didn't manually add any of these shortcuts, this was macOS's auto capitalization at work

\Mathfraka to \Mathfrakz: 𝔄 𝔟 ℭ 𝔡 𝔈 𝔉 𝔤 ℌ ℑ 𝔧 𝔎 𝔏 𝔐 𝔑 𝔬 𝔓 𝔔 𝔯 𝔖 𝔗 𝔲 𝔙 𝔚 𝔛 𝔜 𝔷 -- My disappointment is immeasurable

The last two are based on the other repo author's idea: "One solution is to manually remove the uppercase versions from the listbox, as macOS automatically converts the replacement to uppercase when the first letter you type is capitalized." Removing all the \barA \barB, which I didn't do, would be necessary to allow \bara \barb to work. However, even if it DID work for \mathfrak like it did for \bar, I realized it could still only ever be a partial solution, as we have things like \rightarrow and \Rightarrow that don't have an lower to uppercase relationship (probably true for \mathfraka and \mathfrakA too, as far as macOS knows. Tho I wonder if the relationship is defined in Unicode?)

Sub-bug: dictionary.txt and thus dictionary.plist actually defines each of those twice: \rightarrow on lines 250 and 252 (identical, both →), and \Rightarrow on lines 384 and 547 (conflicting, ⇒ vs ⟹, I believe ⟹ should be assigned to \implies instead). I thought one would simply get overwritten, but no, both appear in the list: image
I shouldn't be surprised it allows a one-to-many mapping tho, as I set a seemingly pointless omw → omw replacement just to override the omw → On my way! replacement that gets added back with each iOS update no matter how many times you delete it... but I digress. I also caught \phi as both φ and ϕ; the latter should be reassigned to \varphi (currently also φ). There are probably a few more duplicates like this, conflicting or not, that probably just slipped through the manual code review. Maybe create an automated script to check for such duplicates/conflicts? I'll open a new issue about this I see there is one already)

The last thing I tried was deleting the capital version of a shortcut (I arbitrarily I chose \mathscrK, its symbol is easy to visually distinguish from \mathscrk), and manually adding \Mathscrk to replace it, JUST in case the first letter theory still works (Greek lowercase letters should tell us it doesn't, but better check). Originally, both \mathscrk and \mathscrK produced 𝒦. After deleting \mathscrK, \mathscrk produced 𝓀, as expected, but so did \Mathscrk (macOS didn't auto-capitalize 𝓀 into 𝒦, as other repo author would've hoped). After manually adding \Mathscrk for 𝒦, we're back to both \mathscrk and \mathscrK producing 𝒦. I then even deleted and added back \mathscrk, in case it's recency of import/modification that determines priority, but nope, both still give 𝒦.

Oh I also tried turning off the macOS global auto-capitalization switch in keyboard settings but that didn't seem to affect anything.

  1. (Minor) Text Replacement doesn't work in all apps on macOS, most notably for me Firefox, which I'm currently using (incidentally, that did make this comment infinitely easier to type). https://discussions.apple.com/thread/251199387 I already knew about that, but others may not, so I point it out in case they think the plist import didn't work if testing on an unsupported app (I myself forget briefly, and thought the same). It works everywhere shortcuts normally work on macOS (including all built-in apps), and potentially universally in all apps on iOS (don't sure though, just based on my experience).

At this point, I'm a bit stumped, and short of talking to an Apple engineer who has access to the code behind how text replacements are done, I'm not sure I can figure out either why the capitalization shenanigans are happening, or how to fix it. If other folks can try importing the plist to test and try to replicate the behavior I saw, that would be helpful.

I'm almost positive this issue is precisely why in https://github.com/clarkgrubb/latex-input: "The Mac LaTeX input source is case insensitive. In LaTeX, \Delta is used for Δ and \delta is used for δ, but the Mac LaTeX input source will nevertheless insist that you disambiguate. It does this with a pop-up window. Use the arrow keys to select the correct symbol and press RETURN." (The pop-up is exactly like typing Chinese/Japanese, which I'm used to.) That repo was also last updated in 2017, and it took some fiddling to install on the latest macOS, which I already did just last week. So I'm really only involving my Mac because macOS is the only way to import a list to sync to iOS.

But taking a step back for a second, how often am I (or is anyone for that matter) going to use \mathscrk and \mathfrakl? As for the Greek and bar letters (and probably some others macOS recognizes upper/lower pairs for), the suggested workaround of deleting the upper shortcut and typing e.g. \Beta and \Barm and using auto-capitalization, in the few situations it's needed, works perfectly fine, and I've already done so. And with LaTeX-input, I don't even need this on my Mac, just iOS.

I can understand if you decide not to officially support macOS/iOS in light of my findings, but I still think it could be useful for many people (I'll certainly be using it with said modifications.) Even though LaTeX-input exists, my earlier points about it being not maintained or updated, and less popular, still remain, not to mention the advantages/featurse mentioned in the other plist repo, e.g. not needing to switch input sources. If you do decide to go forward with merging the PR, I'd be happy to help add documentation/known issues, and/or do more testing.

@hhliu0
Copy link

hhliu0 commented Sep 14, 2023

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants