Snippet Pixie 1.4.0 Released

I give up, there's no way to get Snippet Pixie to auto-expand snippets while typing in non-accessible applications such as browsers or those developed with Electron etc.

I tried monitoring keystrokes at a higher level for tell-tale snippet triggers, then faking keystrokes to backtrack select characters and check the select clipboard for an abbreviation match. This turned out to be a rather resource intensive and flaky method of working out whether a snippet should then be stuffed into the clipboard and pasted over the selection.

I iterated on that method a lot, came up with some routines that made it more and more reliable and performant, but still, some times you had to wait for the checking to do its thing and could often see the selection walking when the error correction and retries kicked in to combat the temperamental async Linux clipboard. If you created any abbreviations that ended in commonly used letters or punctuation characters then you could often find yourself frustratedly waiting for Snippet Pixie to fail so you could carry on typing.

I created a snippet on purpose that I knew would piss me off, it ended in a "$". I did this to drive my desire to improve the algorithm whenever I was writing code for my day job in PhpStorm, you use $ a lot in PHP. It worked to some extent, it caused me enough headaches that I often ended up right clicking Snippet Pixie's icon and hitting "Stop Snippet Pixie" so I could get on with my day. This would impact other areas of my day as I would seriously miss not having a text expander while answering support requests etc. Just silly things, like not wanting to type "WP Offload Media" over and over again, "ome`" is way shorter and easier to type.

A little over a fortnight ago I was using the most excellent Ideogram app by Cassidy James Blaede to pop open an emoji picker and pick ๐Ÿ‘๏ธ for the millionth time that day, when it occurred to me that maybe I should just give in, stop fighting non-accessible apps, and also create a shortcut popup window to search for and paste snippets. So that's what I've done.

Snippet Pixie's Search and Paste Window

You can open Snippet Pixie's "Search and Paste" window at any time by hitting its shortcut, which by default is Ctrl+` (grave).

You can then just start typing the abbreviation or anything you remember from the abbreviation or its contents to narrow down the list of snippets. Hit tab to focus the list, and then either navigate the list with the usual up and down arrows etc and hit Enter on the one you want to use, or hit its number. You can just press Escape or the Ctrl+` shortcut again if you change your mind and don't want to paste a snippet.

When you do choose a snippet from the Search and Paste window it'll close and the expanded text will be sent to the system clipboard and then pasted into whatever app you had open before hitting the shortcut.

It works really well, and the window and general usage pattern may look really familiar to some people. That's because a lot of the code for the Search and Paste window and how it safely closes and pastes is adapted/liberated/borrowed/stolen from Clipped by David Hewitt.

Until I started this little exercise a couple of weeks ago I'd never actually used Clipped before, but had used some of its code when trying to get Snippet Pixie compatible with browsers etc previously. I was led to that code via comments in Ideogram's own code that first time, it worked well for what I needed those bits to do. When I started thinking about how to do a popup snippet search and paste window I first took a look at Ideogram, but that quickly led me to Clipped again. This time I took a much deeper look at the app's code, and had a play with it, it's fantastic! As much as I love the convenience of snippets, I'd not really used a clipboard manager before, but now that I've got Clipped I'm a convert, it's saved me having to go back and re-copy and paste stuff many times.

Anyway, back to Snippet Pixie. It wasn't all plain sailing, but I was able to adapt a good chunk of code from Clipped for the new search and paste window, sprinkle in some code for working with snippets, showing their abbreviations and bodies, and a few little tweaks here and there, and Bob's your uncle, we have a search and paste window opening from a shortcut. ๐ŸŽ‰๏ธ

While I've now been able to rip out all the janky clipboard code I was using for searching entered text for an abbreviation to auto-expand, auto-expansion is still in place for accessible apps. So if you're a fan of Quilter, LibreOffice, or any other native Linux app that is accessible, then you're all good, type your abbreviation and Snippet Pixie will expand it in-place for you. You don't have to use the shortcut.

However, if you decide that you prefer the shortcut, or maybe still have issues with auto-expansion in accessible apps ๐Ÿ˜ฑ๏ธ, then Snippet Pixie now has a new "Auto expand snippets" setting you can toggle on and off in its revamped preferences menu, which has also gained a "Shortcut" submenu.

Snippet Pixie's New Preferences Menu

Oh, yeah, the shortcut, that too was initially inspired by some of Ideogram's code, but I also found some interesting stuff in Clipped regarding being able to change the shortcut. In Snippet Pixie's Shortcut preferences submenu you can change the shortcut from Ctrl+` to whatever suits you best. Just click on the existing shortcut and you'll be asked to type a new one.

Snippet Pixie's New Shortcut Preferences Menu

In the above screenshot you can also see a couple of settings related to how the shortcut works.

You can turn on "Search selected text" if you like. When on, if you have any text selected in an app before you hit the Snippet Pixie search and paste shortcut, then that will be used as the initial search for snippets. It's off by default for a reason though. While at first I thought it was an awesome idea, in reality I often found that the search text was being populated with text I didn't expect, such as previously copied text. This is an unfortunate side effect of how the Linux clipboard works, it's quite easy for the "select" (a.k.a Primary) clipboard to end up with the contents of the copy clipboard. That kind of makes sense when you think about how you first select text and then copy it, but it is a bit annoying some times.

However, if you also use the on by default "Focus search box" setting, then it's not a huge deal as you can just start typing to replace what was automatically inserted into the search box. This is a diversion from how Clipped works, which naturally selects the list of previous clipboard entries as you're very likely to want something just recently clipped and therefore it's more convenient. But with Snippet Pixie I find I'm generally searching for an abbreviation anyway, so by default I've switched initial focus to the search box. However, turn off that "Focus search box" setting and the first entry in the list of snippets will be selected when the window opens. This does have its attractions too when "Search selected text" is enabled as you could find you've got a narrow list of snippets to hand and can quickly select the one you want without having to hit the tab key to focus the list first. ๐Ÿคทโ€โ™‚๏ธ๏ธ

So that's it really, Snippet Pixie 1.4.0 is now available on the elementary OS AppCenter, and will be available on the Snap Store soon¹.

Here's the changelog for Snippet Pixie 1.4.0:

  • Added Search and Paste window, opened with shortcut Ctrl+` by default.
  • Added "Auto expand snippets" checkbox to preferences menu for enabling/disabling snippet expansion while typing in accessible apps.
  • Added ability to change Search and Paste shortcut in preferences menu.
  • Added preference for whether text selected before using shortcut is used for initial search.
  • Added option to focus search box when using the search and paste shortcut.
  • Fixed support for Wayland.
  • Removed auto expanding of snippets in non-accessible applications such as browsers and electron apps (use shortcut for search and paste window instead).

Enjoy!


¹ I'm holding off on the Snap just now as there's a couple of improvements in 1.4.1 that are important to Snaps, once through review and test I'll promote it to the Snap's stable channel.

There's a new version of WP Cron Pixie! ๐ŸŽ‰

WP Cron Pixie 1.4.1 changelog:

  • Fixed wrong data refreshing into non-primary subsite of directory multisite
  • Minor updates to framework and build tools.
  • Tested with WP 5.4

Enjoyed attending #dttech last night, my first @DigitalTaunton event.

Liked the format of 2 main talks + 3 lightning talks after a break.

Great talks, and very well organised event, a lot of work must have gone into it, congrats to all! ๐Ÿ‘

Snippet Pixie 1.3.1 Released

This week I released Snippet Pixie 1.3, with zero new features, but it took many many hours of development.

Umm, what? How could you bump Snippet Pixie up a version, and claim to have spent "many many hours" on development, but not have any new features?

And isn't this post titled "Snippet Pixie 1.3.1 Released"?

Well, take a look at the gif attached to this post, that's a snippet expanding in Standard Notes, an Electron based application. ๐Ÿ˜ฑ๐ŸŽ‰

Demo of Snippet Pixie working with Standard Notes (Electron)

Ok, to some people that might not mean anything, but believe me, it's a major relief to have Snippet Pixie able to work with Electron based apps, and of course, Chrome and Chromium too.

Before v1.3, Snippet Pixie relied on apps using the ATK library, or at the very least, doing what's needed to work with the accessibility DBus interfaces that AT-SPI interacts with. Further still, the user had to be typing in an EditableText and Text interface compliant control. There are many Linux applications that are ATK/AT-SPI compliant, but unfortunately a few very popular ones are not.

The biggest trouble maker for Snippet Pixie was Chrome, turns out it's quite a popular browser. ๐Ÿ˜‰

And many companies build cross platform apps with Electron, a technology that basically uses Chrome under the hood, so they weren't working well with Snippet Pixie.

I tried all kinds of things to try and get Chrome and Electron apps to play nice, and there was the odd occasion when a control here and there would actually work with the accessibility framework, but in the end I had to throw out the accessibility stuff and go "old skool".

Well, I say "throw out the accessibility stuff", but that's not quite true, and is the reason for v1.3.1, but we'll come to that in a minute.

Snippet Pixie still uses AT-SPI for monitoring keystrokes, but it's using a method that is a little more greedy in its scope, monitoring "all windows" for the current app rather than just the currently focused text control. With this method, Snippet Pixie can see when a key is pressed that matches the last character of some snippet's abbreviation, and then go look to see whether previously entered text matches an abbreviation.

This may sound less performant than monitoring the currently focused text control, and technically it likely is, but, it turns out that with this method I was able to eliminate a huge performance problem that earlier versions of Snippet Pixie suffered.

Snippet Pixie used to have to notice when the current app changed, then go look through all the widgets in that app to try and find the currently focused one, which hopefully was an editable text control. When that app happens to be a spreadsheet with thousands of widgets, that search for the currently focused control (cell) could take a noticeable amount of time. Now, we're not talking minutes here, not even seconds, but if it took even anything near half a second, you'd notice it.

Because Snippet Pixie is now monitoring at a higher level by default, it does not need to go look for a widget to attach monitoring code to. That's a big boost in performance, it no longer feels like Snippet Pixie is slowing down app switching for some apps.

Hang on a minute, if you no longer know where the text is being entered, how are you checking whether an abbreviation precedes that key stroke that matches the end of one or more abbreviations?

Blimey, you do ask some very intelligent questions don't you? ๐Ÿ˜‰

So this is where a huge amount of time was spent in developing this release. When I first thought of building Snippet Pixie, one of the initial ideas I had for how to get the just entered text was to see if there was a way to walk back through the text with some sort of selection method, and then check the contents of the selection. I didn't get too far in that investigation before finding the accessibility interfaces, which seemed like a much saner approach.

Recently I found out about another open source text expander style app called Espanso. There was an issue raised on the project about the clipboard being overwritten when an expansion happened, or something like that, can't quite remember. But anyway, seeing that made me think that maybe I should take another look at my mad idea of selecting text, stuffing it into the clipboard and then checking the contents of the clipboard.

After much research and experimentation, I eventually managed to get Snippet Pixie to start a text selection by "pressing" the shift key and then "pressing and releasing" the left cursor key, then fake a copy shortcut, check the contents of the clipboard, and then keep hitting the left cursor key until an abbreviation was found, or the max length of known abbreviations was exceeded.

This kind of worked, and once I had an abbreviation match it was a cinch to then stuff the expanded text into the clipboard and fake a paste hotkey. Thanks for the initial code Clipped!

Grabbing the selected text into the clipboard wasn't ideal though. It was very flakey, using the copy shortcut for every extra character was hit and miss, and there was some weirdness with the clipboard not always updating.

I learnt a lot about the async mechanisms that the Linux desktop (GTK+ based in this case) clipboard uses, and that there's such a thing as a clipboard that most people think about, but also a second selection clipboard that auto populates when text is selected.

With that knowledge to hand, and after much experimentation with threads and strategic yielding and microscopic usleeps (I hate handling threads, but who doesn't?), I finally got a nice mechanism that recognises the abbreviation via selection, and then expands via the clipboard.

Over time I fine tuned the mechanism, and added a bunch of speed ups and error correcting retries to the way abbreviations are searched for, and ensured that the current clipboard is saved away and restored before and after the expansion respectively. The attached gif is intentionally slow so you can see the walk back working, even error correcting, but in general the mechanism is way quicker.

It worked really well with Chrome, Chromium and even Electron apps, but when I tested LibreOffice Writer, which had worked fine with the previous version of Snippet Pixie, it did the most weird things and lost text selection all the time, making abbreviation recognition impossible.

So I brought back the core abbreviation recognition engine of the old accessibility framework dependent mechanism, and sure enough, that still worked with LibreOffice Writer. Well, it wasn't quite that simple, I also had to add back a mechanism for being notified of when an accessible control was focused, and do a bunch of work to dynamically swap between the "all windows" monitoring method and the control based one as appropriate when apps are switched.

I did not need to re-add the old widget lookup code though, and I was able to improve the accessibility based mechanism with the abbreviation lookup speedups and threading from the new text selection mechanism.

In the end I got to a place where the previously supported applications still worked with the faster accessibility based text editing, and added support for a much wider set of applications via the new text selection mechanism.

When I applied those speed ups to the old mechanism that I created while developing the new one, I did miss one problem that necessitated v1.3.1. I used a slightly different monitoring mode with the re-implemented accessibility based code, async rather than blocking, kinda needed as part of the performance improvements and thread usage. Sometimes the control where the text was being entered would report the caret position incorrectly, especially near the beginning of the control, and after initial switch into the app. Turns out I needed to yield the abbreviation checking thread for a split second to let the app get itself together.

There are unfortunately a few apps that remain incompatible with Snippet Pixie 1.3.1.

The one that hurts the most for me is Firefox, my preferred web browser, which at some point in its last few releases started to misbehave when Snippet Pixie was in use. The old accessible method that Snippet Pixie used stopped working for some reason, something must have changed in Firefox to stop keystroke monitoring. I wouldn't be surprised if this was an intentional security measure, but I bet it also stops screen readers from working now.

When forcing use of the new text selection mechanism with Firefox, it keeps losing focus while you're typing, which breaks everything, text stops being entered. To get around this problem temporarily, until I can find a proper solution, or Firefox gets fixed, Snippet Pixie now blacklists Firefox so at least Firefox is usable, even if snippets don't expand.

Similarly, Snippet Pixie has never worked with terminal emulators, their text handling mechanism is fundamentally different. To avoid any potential issues with the new text selection mechanism that Snippet Pixie uses, a bunch of popular terminal emulators are now blacklisted by Snippet Pixie and it turns off abbreviation checking while they're the active application.

This new blacklisting mechanism actually brought to the fore another performance improvement for Snippet Pixie, as turning off the abbreviation checking speeds things up in incompatible apps. And along with this change, Snippet Pixie also monitors less keystrokes in general now too anyway, as if you're using either the Ctrl or left Alt key, as far as I can tell you can't be entering a character. The right Alt Gr key is still monitored in combination with normal keys though as that's how you generally enter extended characters such as โ‚ฌ and ยข.

The final incompatibility with Snippet Pixie 1.3.1 is Wayland compositors. This too hurts as I'm a fan of Sway, but alas the way I've had to implement active window / app checking is via libwnck, which unfortunately only supports X11.

I'm actively trying to find either an alternative robust method of tracking the current application that works in X11 and Wayland, or an additional Wayland only method that I can use when Wayland is detected.

So I guess you could maybe class the expanded compatibility and vastly improved performance as new features, but even if you don't, I'm sure you can see why I bumped the version from 1.2.x to 1.3.x.

Anyway, that was a bit of a rambling retrospective into what went into Snippet Pixie 1.3.1. Hope you enjoy this release, please submit bugs and feature requests in the GitHub repo.

Get

Get

FYI, if you try to circumvent paying for support for a client's product I work on by bombarding my personal email address, my contact page, or any other means that are not a proper support channel, I'm still not going to reply to your vague mess of a support request! ๐Ÿ‘ฟ

TIL: enabling boot.cleanTmpDir on NixOS can solve the most head-scratchy of head-scratchy things!

Yay, after a couple of days painting the landing I'm finally at podcast zero! ๐ŸŽ‰ (there may have been a good bit of podcast culling and skipping too)