Calendar progress

As we’re closing in on a simple but functional calendar for Kube, I’d like to share our progress with you.

We’ve decided to start with a week view, as that seems to be a good compromise between information density and enough information for day-to-day use.
We will eventually complement that with a month view, which is probably all we need for the time being.

An agenda view will probably rather become part of a separate view, showing you upcoming events, tasks that need to be taken care of, important emails, ….

Anyhow, here’s the week view:

Screenshot_20180823_103304

This view is based on two models, one for full-day events (on top) and one for the rest below. The models each query for all events that overlap with the time-range shown, which we can do efficiently due to the work of Rémi on the calendar queries. That means we really only have to deal with a few events in memory, which we can easily do on the spot. The models then do their magic to calculate the position and sizes of all the necessary boxes, so all we have to do in QML is paint them. This now also includes recurring events, although we’re not dealing with exceptions just yet.

The colors of the events are taken from the calendar, which gets synchronized via CalDAV. This is a tradeoff design wise because you can bet that those colors will not match Kube in any way. But we decided that Kube should work well with other devices and clients, and the color is a major factor to recognize immediately what belongs where. Perhaps we can still make the colors a bit easier on the eye by desaturating them a bit, we’ll have to see.

Otherwise there is dimming of the past, and a blue line indicating the current time. Simple and without distractions.

The next steps are going to be adding a detail view, as well as a simple editor, and then we should already have the basics for your daily needs.

We’re still running a Kolab Now promotion for Kube! For more information head over to: blogs.kolabnow.com/2018/08/29/kube-campaign-last-chance

Advertisements

Optimizing Kube’s storage

Near the middle / end of my internship, I got to modify parts of the storage system in Sink, the library handling all the data of Kube.

The goal was to both to speed up the storage and reducing disk space. These two goals often go hands in hand in databases, since smaller data means faster disk lookup, and more data put in memory, available for direct usage.

Reducing key sizes

The first important modification was for keys to be stored in a binary format instead of the displayable format. This allowed data retrieval to be faster, because the binary format would be much smaller than the display format.

On the memory usage side, this is a bit more awkward to measure: on the one hand, the since the size of keys is smaller in memory, but on the other hand, LMDB (the database system we are using) will try to put more data in memory.

But on the whole, either we are using less memory, or Kube will be faster altogether. The reality is probably a mix of the two.

As for the numbers, these are the results of two different benchmarks:

Develop branch This patch
Current Rss usage [kb]: 40700 Current Rss usage [kb]: 39112
On disk [kb]: 10788 On disk [kb]: 8836
Write amplification: 12.0075 Write amplification: 9.83485
Develop branch This patch
Total pages: 760 Total pages: 603
Used size: 1425408 Used size: 1191936
Total on disk: 3293184 Total on disk: 2650112
Write amplification: 3.63268 Write amplification: 3.51402

As we can see on both tables, we use less disk space after the patch, but memory usage has not gone down in all places.

Overall we use approximately 20% less disk space.

Separating UIDs and revisions

Another important modification on the storage system was separating UIDs and revisions. Before this patch, we used to store the UID and the revision as keys inside the database.

What’s more is that the key was in the format “{this-is-a-uid}0000000000042”. The reason for padding the number with zeroes was that we need the data to be ordered, and we needed the keys to be stored as a string for the UID.

However, every revision is unique, so we can store only the revision as a key. This also allows us to store the key as a number, use the “integer key” feature of LMDB so the sorting stays correct and save a nice amout of space.

But, since we can keep old revisions, we need to track them. Therefore we need another table for mapping UIDs and revisions.

The rationale behind this patch is that since the “uid to revisions” database will be much smaller than the main database, it can be put into memory, leading to faster lookups.

Even though the results were not as consequent as the “reducing key sizes” patch, it seems we still got a small improvement performance wise. We will better see how different the performance is in the near future, though.

In the meantime, we can use the benchmarks graphics to see how performance, memory and disk usage was impacted (courtesy of our CI):

Disk usage with dummy writes
Disk usage with dummy writes
Write amplification
Write amplification
Memory usage with dummy writes
Memory usage with dummy writes
Initial query time
Initial query time
Pipeline performance
Pipeline performance

Continue reading “Optimizing Kube’s storage”

Last week in Kube

  • Lot’s of progress on the calendar. We now have a nice little read-only calendar. There’s still important stuff missing, like recurrences, but we’re getting there and I’m already using it with my actual calendar data.
    • A date switcher to move between weeks.
    • We now show the date for each day and the week number for the week view.
    • The events get the proper colors from the calendar.
    • The view can be filtered by calendar using the checkbox.
    • The calendar checkbox now indicates the color, to save some precious space in that area.
    • A tooltip provides the full calendar name if there is not enough space anyways.
    • The calendar list is now scrollable with many calendars.
  • Fixed the name in the menu and dock as well as the mounted image name on Mac OS.
  • Fixed attachment dialogs on Mac OS.
  • The CI now checks the intended download paths for availability.
  • The website now has build status badges from the CI.
  • The “New Email” button (and similar green buttons) now has improved focus indication.
  • We’re now fixing the minimum font size to 11 if there is no QT_QPA_PLATFORMTHEME defined on linux. Qt hardcodes it to 9 otherwise, which is tiny.
  • Because flatpak uses pid-namespaces and lmdb uses pid’s for locking, starting two flatpaks results in a crash (the db is shared but the pid’s are guaranteed to be the same). It doesn’t seem like there is a good solution to this short of communicating with the original flatpak and ensuring all related processes are running in the same container. For now this is circumvented by simply not starting the second instance using a lockfile.
  • We removed the IMAP and similar protocol specific accounts and replaced it with a single “Custom” account. The account concept is supposed to encapsulate multiple procols, such as IMAP, SMTP, CalDAV and CardDAV as one logical unit. It thus doesn’t make much sense to have an IMAP specific account, and with the new “Custom” account you can still set-up IMAP only, or combine it with CalDAV and CardDAV so you can use all aspects of Kube.

Screenshot_20180823_103304

Kube Commits, Sink Commits

Previous updates

More information on the Kolab Now blog!

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube-project.com

Last week in Kube

  • We have a new filter overlay. We opted to avoid the inline searchbar entirely, and are using the overlay to display it instead.
  • We try harder to display all search results and not artifically limit it. This was especially problematic because we don’t sort the search results before the limit, meaning we could end up loosing very relevant search results (such as recent mails).
  • The composers html switch is gone. Instead we simply always offer the buttons to apply formatting, together with a button to remove all formatting for plaintext instead.
  • Rémi landed his storage improvement patch which shaves off a cool ~15% of storage requirements. This also helps performance because smaller db mean less data to load into memory. This was followed up by fix for a performance regression caught by the CI.
  • The Kube icon is now available in all necessary sizes to make it look good on Mac.
  • Important emails are now indicated in the maillist.
  • The default settings of the GMail account have been fixed.
  • There is finally a separate login and email address field in the IMAP account configuration.
  • We have a new website at kube-project.com where we now also offer the nighlies. These are now real nightlies that are automatically updated if all CI checks pass.
  • Prevented multiple flatpak instances at the same time as this does not currently work properly.
  • Last but not least, there’s progress on the Calendar.
Screenshot_20180816_165028
Never mind the colors, they are coming from the CalDAV backend.

 

Kube Commits, Sink Commits

Previous updates

More information on the Kolab Now blog!

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube-project.com

Kube: new website, new flatpak

Kube has a new website: kube-project.com

Screenshot_20180806_172259

It’s got a fresh, cleaner design, together with less, but hopefully more to the point, content.

What comes with it though is that we’ll also be publishing the flatpak and Mac OS nightlies there from now on. The CI that is building those nightlies will be integrated eventually, but that job is not complete just yet.

So update your bookmarks now, going forward kube-project.com will be the first stop for anything Kube.

Kube 0.7.0 is out!

I’m pleased to announce the immediate availability of Kube 0.7.0

Over the past year or so we’ve done a lot of work and building and maturing Kube and it’s underlying platform Sink.
Since the last publicly announced release 0.3.0 there have been 413 commits to sink and 851 to Kube. Since that diff is rather large I’ll spare you the changelog and will do a quick recap of what we have instead:

  • A conversation view that allows you to read through conversations in chronological order.
  • A conversation list that bundles all messages of a conversation (thread) together.
  • A simple composer that supports drafts and has autocompletion (assisted by the addressbook) for all recipients.
  • GPG support for reading and writing messages (signing and encryption).
  • Automatic attachment of own public key.
  • Opening and saving of attachments.
  • Rendering of embedded messages.
  • A read-only addressbook via CardDAV.
  • Full keyboard navigation.
  • Fulltext search for all locally available messages.
  • An unintrusive new mail hint in the form of a highlighted folder.
  • Kube is completely configuration free apart from the account setup.
  • The account setup can be fully scripted through the sinksh commandline interface.
  • Available for Mac OS.
  • Builds on Windows (But sadly doesn’t completely work yet).
  • The dependency chain has been reduced to the necessary minimum.

While things still change rapidly and we have in no way reached the end of our ever growing roadmap, Kube has already become my favorite email client that I have ever used. YMMV.

Outlook

Turns out we’re not done yet. Among the next plans we have:

  • A calendar via CalDAV (A first iteration is already complete).
  • Creation of new addressbook entries.
  • A dedicated search view.

While we remain committed to building a first class email experience we’re starting to venture a little beyond that with calendaring, while keeping our eyes focused on the grander vision of a tool that isn’t just yet another email client, but an assistant that helps you manage communication, time and tasks.

Tarballs

Get It!

Of course the release is already outdated, so you may want to try a flatpak or some distro provided package instead:

https://kube.kde.org/getit.html

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube.kde.org

Last week in Kube

Last week in Kube

  • Kube builds on Windows and is largely functional. There is still an issue with Xapian not working and LMDB sparse files being broken on windows.
  • Date-range queries have been implemented.
  • The flatpak’s gpg-agent integration has been “fixed” (The proper fix will have to be done on the flatpak side of things).
  • A bunch of bugs in live-query updating have been fixed.
  • We now do some basic conflict resolution to avoid overwriting local changes with changes from the server.
  • Various visual glitches and and keyboard navigation issues have been fixed.
  • The generated message-id for new messages no longer leaks the local hostname.
  • Added tooltips for various UI elements.
  • Various message parser fixes, especially for apple mail generated messages with attachments.
  • The Logview has been renamed to Notifications View, and now only shows up if it also contains something.

Kube Commits, Sink Commits

Previous updates

More information on the Kolab Now blog!

“Kube is a modern communication and collaboration client built with QtQuick on top of a high performance, low resource usage core. It provides online and offline access to all your mail, contacts, calendars, notes, todo’s and more. With a strong focus on usability, the team works with designers and UX experts from the ground up, to build a product that is not only visually appealing but also a joy to use.”

For more info, head over to: kube.kde.org