I’ve spent the last few days with fellow KDE hackers in beautiful Randa in the Swiss Mountains.
It’s an annual event that focuses on a specific topic every year, and this time accessibility was up, so Michael and me made our way up here to improve Kube in that direction (and to enjoy the scenic surroundings of course).
I have to admit we didn’t spend all our time on accessibilty, since we’ve recently done a big push in that direction already with keyboard navigation and translation preparations. Anyhow, here’s an overview on what we’ve worked on:
Query performance
While it might now be a showstopper, there is an annoyance in Kube that if you initially sync a lot of mail (my INBOX alone contains 40k) this not only uses a lot of CPU in the synchronizer process (which is expected), but also in the Kube client process, which seems odd at first.
The reason for this are the live queries, which update themselves whenever the resource signals that it has a new revision available (A new revision could be a new mail or a modification for instance). Usually, when we process an update we do so incrementally. If you just received a new mail or two, or something has been marked as read etc., that is also the sensible thing to do. It’s a comparatively tiny amount of work, and we really want incremental changes in the result anyways, otherwise we’d mess up the client’s UI state (selection, scroll position, …).
However, when we fetch 40k mails that also means we get 40k updates that we have to process and it’s surprisingly hard to avoid that:
- We don’t know in advance that there will be 40k updates, so we have to process the updates initially.
- We can’t really skip any updates, because any update might affect what we have in the view. We just don’t know until we’ve already processed it.
- We can’t just fall back to redoing the full query because we need incremental changes, otherwise you’d see a blinking UI until the sync is done.
- We don’t want to just stop updating until the sync is done, because in most cases you already have useful data to work with, so this is really a background task, and we don’t want to block all updates.
If you end up processing the larger part of the data-set as incremental update it becomes naturally really expensive, because you can’t leverage the indexes which made queries efficient in the first place. So this is not something that can just be optimized away.
This means some middleground has to be struck which allows for updates, but doesn’t render the application maxed out because it’s updating so fast.
A possible approach to fix this would be the following:
- Set some threshold when it becomes cheaper to redo the query instead of processing the updates.
- If the threshold is reached, redo the query from scratch.
- Diff the current results with the redone results so we can still update the UI incrementally.
- Apply some event compression to the revision updates, so we land in the optimized path quicker.
Given that this really isn’t a huge problem right now, since the application does remain responsive (querying is done in a separate thread), and this will only ever happen if you sync a large folder for the first time, I won’t spend a lot of time on that for now. The most important part was figuring out why this problem appeared in the first place, and understanding it’s reasons and implications. This will allow us to attack it properly once we get to it.
HTML composer
Kube’s composer is currently a simple text field where you can type plain text. This is obviously not the end goal.
To explore our options in providing a more advanced composer, we tried prototyping a simple HTML editor.
Unfortunately our experiments showed that QML isn’t really up to that yet. The official example falls flat on it’s face when trying it (it simply fails except for a few specific cases), and the “API” to do any sort of formatting is more a hack than anything else.
Granted, the API docs of the TextArea also state that you can’t modify the internal document, or in other words, no HTML composer for us.
The options we have left are:
- Some custom text editing component (sounds like a lot of work, probably is).
- Try to hack around TextAreas deficiencies until we find some magic combination that works.
- Use a WebView and implement everything in javascript (now that’s just sad).
For now the item is postponed.
GPG Support
We’ve scoped out what we want to reach for initial GPG support in Kube. While we already have read-only GPG support (we can decrypt mails and verify signatures), there is some more work to be done until we can also sign and encrypt messages we’re sending. The focus will be on keeping things simple to get the job done, and not build the most advanced GPG interface ever.
Things that will happen are:
- Key selection for the account that will then be used for encryption.
- Key setup for new accounts that don’t have a key yet (either by importing an existing key or creating a new one)
- Encryption option in the composer that depends on the availability of keys for all recipients.
- Key selection for users in the addressbook.
All of it can be found on Phabricator
Cross platform builds
Since we had some cross platform building experts on site, I explored some of our options:
- Cross compiling for Windows with MXE.
- Building on Windows and OSX using craft.
- Building with some custom scripts on Windows.
- Building with macports on OSX.
- Building with brew on OSX.
Although I had mixed feelings about craft back from the days when I used emerge (the craft predecessor) to build Kontact on Windows,
it at least looks like the most sensible option still.
Crosscompiling using MXE seems ok-ish for Windows; I had some success for parts of the stack and the rest will just be the grunt work of making tarballs and writing build definition files. At least that way you don’t have to work on a Windows system. However I heard that the compiled end-result will just not be as good as using native compilers, so perhaps it’s not worth it to trade some building convenience for a sub-par end result.
Also, writing the definition files is pretty exactly the same as for craft, and MXE would be for Windows only.
So, looks like I’ll be doing another round with craft sometime soonish.
Plasma phone
It kinda works.
Bushan Sha quickly wipped up a package for the Plasma Phone and installed it, which allowed us to see Kube running for the first time on a mobile device, which felt surprisingly cool 😉
There clearly are some rough edges, such as the UI being completely unsuitable for that form factor (who would have thought…), and a bug in one of the QML components which breaks folder selection using a touch screen. Still, it was nice to see =)
Input validation
To improve the UX with forms in Kube, or more specifically the configuration dialog, we wanted to provide some feedback to the user on what fields need to be filled out. Michael implemented a neat little visualization that that marks invalid input fields and gently fades away once the input becomes valid.
So far we only validate that fields are not empty, but that can be changed by assigning new validators that e.g. check that an email address field indeed contains a valid email address.
Recap
It was once again a very productive week, with lot’s of good food and a couple of fun hikes in between to clear out the head.
Besides the usual work going on it’s invaluable to have an exchange with other people that just might have solved the problem already that you’re going to spend the next few weeks on. So in many ways this is not only about a concentrated work effort, but also about professional training and exchange, and that is only possible in when you’re sitting together with those people.
I’d like to thank the whole team that organized the sprint, took care of us with great food and lot’s of chocolate and generally just made our stay in Randa great once more.
If you made it all the way down here and skipped the fundraiser on top, please consider donating something, it’s the only way we can keep doing meetings like this and they are invaluable for us as a community.
An obvious reason why we picked Kolab Now is because it is what sustains the larger part of the Kube team, allowing us to work on this project in the first place. However, it’s also a prime example of a completely Open Source and standards compliant service. Improving the Kolab experience means improving IMAP support, improving the CardDAV implementation, perhaps even adding CalDAV. It also means implementing proper GPG support, and pushing the user experience edge by edge to where we expect it to be. Things that all standards compliant services will benefit from. The Kolab Now service ensures we can focus on the relevant problems by taking variables out of the equation by being essentially the reference installation of Kolab.
