Last months in Kube

Kube is still alive! I got distracted for a while, both professionally and privately, and writing blog posts is unfortunately always the first thing that ends up on the chopping block. Anyways, lot’s of progress has been made (103 commits in sink, ~130 commits in the kube codebase):

• For sink we had a variety of bugfixes and performance improvements, especially on the more recent CalDAV/CardDAV backends.
• For CalDAV/CardDAV we now do basic autodiscovery using the .well-known url’s. The DNS part of the spec has not been implemented so far.
  This means for a properly set-up server you only have to specify the base url, and everything else will be discovered automatically from there.
• On the more user-facing front we have:
  • Sent emails are now collapsed by default
  • Plain text is now the preferred method of viewing emails. You can still view the HTML variant if available by clicking a button.
    • The Addressbook is no longer read-only and you can now create contacts as well.
  • A visually reworked composer that avoids becoming too wide and removes a lot of the visual clutter.
  • The calendar can now render recurring-events.
  • It is now possible to create events as well.
  • Work on a tasks view has started

Conversations

Composer and drafts

Composer and recipients

Addressbook

Calendar

Tasks

Releases

You may have noticed that it’s been a while since the last release. This is not only because releases are additional work, but also because we already have a continuous delivery method with the nightly flatpak.
It’s clear that releases do provide value, both as a communication tool which version should be packaged, and if they would be maintained. With the current manpower we cannot maintain releases though, which makes it significantly less interesting.

With that said, the 0.8 release with the calendar is now long overdue and should be coming out soonish.

Experimental flatpak

Just to put it out there; Additionally to the usual “master” branch of the flatpak, there is also an “experimental” branch, containing, surprise, various experimental bits and pieces.

This currently entails:

• A plugin that stores the accounts password encrypted by the accounts gpg key (blindly assuming there is one with a matching email address).
• A search view
• The upcoming calendar view (which we’ll move over in the next release)
• The above todo view (which will take a little longer to move to master)
• A “File as expense” plugin (a showcase how we could do extensions in the mail view).
• The Inbox crusher view (an experiment for a view to go through your inbox one-by-one).

It typically serves as a staging ground for new components, and is the version that I’m running day-to-day. flatpak makes it easy to switch back and forth between the branches on top of the same dataset, so you can try it and switch back if you don’t like what you see.

To give it a shot use the following command to install and switch to the experimental flatpak branch:

flatpak -y --user install --from https://files.kube-project.com/flatpak/com.kubeproject.kube.experimental.flatpakref
flatpak --user make-current com.kubeproject.kube experimental

To switch back simply issue:

flatpak --user make-current com.kubeproject.kube master

Kube Commits, Sink Commits

Previous updates

More information on the Kolab Now blog!

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.” For more info, head over to: kube-project.com

Advertisements

Calendar progress

As we’re closing in on a simple but functional calendar for Kube, I’d like to share our progress with you.

We’ve decided to start with a week view, as that seems to be a good compromise between information density and enough information for day-to-day use.
We will eventually complement that with a month view, which is probably all we need for the time being.

An agenda view will probably rather become part of a separate view, showing you upcoming events, tasks that need to be taken care of, important emails, ….

Anyhow, here’s the week view:

This view is based on two models, one for full-day events (on top) and one for the rest below. The models each query for all events that overlap with the time-range shown, which we can do efficiently due to the work of Rémi on the calendar queries. That means we really only have to deal with a few events in memory, which we can easily do on the spot. The models then do their magic to calculate the position and sizes of all the necessary boxes, so all we have to do in QML is paint them. This now also includes recurring events, although we’re not dealing with exceptions just yet.

The colors of the events are taken from the calendar, which gets synchronized via CalDAV. This is a tradeoff design wise because you can bet that those colors will not match Kube in any way. But we decided that Kube should work well with other devices and clients, and the color is a major factor to recognize immediately what belongs where. Perhaps we can still make the colors a bit easier on the eye by desaturating them a bit, we’ll have to see.

Otherwise there is dimming of the past, and a blue line indicating the current time. Simple and without distractions.

The next steps are going to be adding a detail view, as well as a simple editor, and then we should already have the basics for your daily needs.

We’re still running a Kolab Now promotion for Kube! For more information head over to: blogs.kolabnow.com/2018/08/29/kube-campaign-last-chance

Optimizing Kube’s storage

Near the middle / end of my internship, I got to modify parts of the storage system in Sink, the library handling all the data of Kube.

The goal was to both to speed up the storage and reducing disk space. These two goals often go hands in hand in databases, since smaller data means faster disk lookup, and more data put in memory, available for direct usage.

Reducing key sizes

The first important modification was for keys to be stored in a binary format instead of the displayable format. This allowed data retrieval to be faster, because the binary format would be much smaller than the display format.

On the memory usage side, this is a bit more awkward to measure: on the one hand, the since the size of keys is smaller in memory, but on the other hand, LMDB (the database system we are using) will try to put more data in memory.

But on the whole, either we are using less memory, or Kube will be faster altogether. The reality is probably a mix of the two.

As for the numbers, these are the results of two different benchmarks:

Develop branch	This patch
Current Rss usage [kb]: 40700	Current Rss usage [kb]: 39112
On disk [kb]: 10788	On disk [kb]: 8836
Write amplification: 12.0075	Write amplification: 9.83485

Develop branch	This patch
Total pages: 760	Total pages: 603
Used size: 1425408	Used size: 1191936
Total on disk: 3293184	Total on disk: 2650112
Write amplification: 3.63268	Write amplification: 3.51402

As we can see on both tables, we use less disk space after the patch, but memory usage has not gone down in all places.

Overall we use approximately 20% less disk space.

Separating UIDs and revisions

Another important modification on the storage system was separating UIDs and revisions. Before this patch, we used to store the UID and the revision as keys inside the database.

What’s more is that the key was in the format “{this-is-a-uid}0000000000042”. The reason for padding the number with zeroes was that we need the data to be ordered, and we needed the keys to be stored as a string for the UID.

However, every revision is unique, so we can store only the revision as a key. This also allows us to store the key as a number, use the “integer key” feature of LMDB so the sorting stays correct and save a nice amout of space.

But, since we can keep old revisions, we need to track them. Therefore we need another table for mapping UIDs and revisions.

The rationale behind this patch is that since the “uid to revisions” database will be much smaller than the main database, it can be put into memory, leading to faster lookups.

Even though the results were not as consequent as the “reducing key sizes” patch, it seems we still got a small improvement performance wise. We will better see how different the performance is in the near future, though.

In the meantime, we can use the benchmarks graphics to see how performance, memory and disk usage was impacted (courtesy of our CI):

Disk usage with dummy writes

Write amplification

Memory usage with dummy writes

Initial query time

Pipeline performance

Last week in Kube

• Lot’s of progress on the calendar. We now have a nice little read-only calendar. There’s still important stuff missing, like recurrences, but we’re getting there and I’m already using it with my actual calendar data.
  • A date switcher to move between weeks.
  • We now show the date for each day and the week number for the week view.
  • The events get the proper colors from the calendar.
  • The view can be filtered by calendar using the checkbox.
  • The calendar checkbox now indicates the color, to save some precious space in that area.
  • A tooltip provides the full calendar name if there is not enough space anyways.
  • The calendar list is now scrollable with many calendars.
• Fixed the name in the menu and dock as well as the mounted image name on Mac OS.
• Fixed attachment dialogs on Mac OS.
• The CI now checks the intended download paths for availability.
• The website now has build status badges from the CI.
• The “New Email” button (and similar green buttons) now has improved focus indication.
• We’re now fixing the minimum font size to 11 if there is no QT_QPA_PLATFORMTHEME defined on linux. Qt hardcodes it to 9 otherwise, which is tiny.
• Because flatpak uses pid-namespaces and lmdb uses pid’s for locking, starting two flatpaks results in a crash (the db is shared but the pid’s are guaranteed to be the same). It doesn’t seem like there is a good solution to this short of communicating with the original flatpak and ensuring all related processes are running in the same container. For now this is circumvented by simply not starting the second instance using a lockfile.
• We removed the IMAP and similar protocol specific accounts and replaced it with a single “Custom” account. The account concept is supposed to encapsulate multiple procols, such as IMAP, SMTP, CalDAV and CardDAV as one logical unit. It thus doesn’t make much sense to have an IMAP specific account, and with the new “Custom” account you can still set-up IMAP only, or combine it with CalDAV and CardDAV so you can use all aspects of Kube.

Kube Commits, Sink Commits

Previous updates

More information on the Kolab Now blog!

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube-project.com

Last week in Kube

• We have a new filter overlay. We opted to avoid the inline searchbar entirely, and are using the overlay to display it instead.
• We try harder to display all search results and not artifically limit it. This was especially problematic because we don’t sort the search results before the limit, meaning we could end up loosing very relevant search results (such as recent mails).
• The composers html switch is gone. Instead we simply always offer the buttons to apply formatting, together with a button to remove all formatting for plaintext instead.
• Rémi landed his storage improvement patch which shaves off a cool ~15% of storage requirements. This also helps performance because smaller db mean less data to load into memory. This was followed up by fix for a performance regression caught by the CI.
• The Kube icon is now available in all necessary sizes to make it look good on Mac.
• Important emails are now indicated in the maillist.
• The default settings of the GMail account have been fixed.
• There is finally a separate login and email address field in the IMAP account configuration.
• We have a new website at kube-project.com where we now also offer the nighlies. These are now real nightlies that are automatically updated if all CI checks pass.
• Prevented multiple flatpak instances at the same time as this does not currently work properly.
• Last but not least, there’s progress on the Calendar.

Never mind the colors, they are coming from the CalDAV backend.

 

Kube Commits, Sink Commits

Previous updates

More information on the Kolab Now blog!

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube-project.com

Kube: new website, new flatpak

Kube has a new website: kube-project.com

It’s got a fresh, cleaner design, together with less, but hopefully more to the point, content.

What comes with it though is that we’ll also be publishing the flatpak and Mac OS nightlies there from now on. The CI that is building those nightlies will be integrated eventually, but that job is not complete just yet.

So update your bookmarks now, going forward kube-project.com will be the first stop for anything Kube.

Kube 0.7.0 is out!

I’m pleased to announce the immediate availability of Kube 0.7.0

Over the past year or so we’ve done a lot of work and building and maturing Kube and it’s underlying platform Sink.
Since the last publicly announced release 0.3.0 there have been 413 commits to sink and 851 to Kube. Since that diff is rather large I’ll spare you the changelog and will do a quick recap of what we have instead:

• A conversation view that allows you to read through conversations in chronological order.
• A conversation list that bundles all messages of a conversation (thread) together.
• A simple composer that supports drafts and has autocompletion (assisted by the addressbook) for all recipients.
• GPG support for reading and writing messages (signing and encryption).
• Automatic attachment of own public key.
• Opening and saving of attachments.
• Rendering of embedded messages.
• A read-only addressbook via CardDAV.
• Full keyboard navigation.
• Fulltext search for all locally available messages.
• An unintrusive new mail hint in the form of a highlighted folder.
• Kube is completely configuration free apart from the account setup.
• The account setup can be fully scripted through the sinksh commandline interface.
• Available for Mac OS.
• Builds on Windows (But sadly doesn’t completely work yet).
• The dependency chain has been reduced to the necessary minimum.

While things still change rapidly and we have in no way reached the end of our ever growing roadmap, Kube has already become my favorite email client that I have ever used. YMMV.

Outlook

Turns out we’re not done yet. Among the next plans we have:

• A calendar via CalDAV (A first iteration is already complete).
• Creation of new addressbook entries.
• A dedicated search view.

While we remain committed to building a first class email experience we’re starting to venture a little beyond that with calendaring, while keeping our eyes focused on the grander vision of a tool that isn’t just yet another email client, but an assistant that helps you manage communication, time and tasks.

Tarballs

• http://download.kde.org/unstable/sink/0.7.0/src/sink-0.7.0.tar.xz.mirrorlist
• http://download.kde.org/unstable/kube/0.7.0/src/kube-0.7.0.tar.xz.mirrorlist

Get It!

Of course the release is already outdated, so you may want to try a flatpak or some distro provided package instead:

https://kube.kde.org/getit.html

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube.kde.org