Build your productivity tools with org-mode

Optimizing my workflow has always been one of my personal goals. A smoother developer experience makes your velocity go through the roof and consequently makes you happier.

It is not, by any means, something easy to accomplish though, especially given the enormous amount of "productivity tools" out there.

I try to follow a few guidelines:

  • use as few tools as possible to avoid context switching
  • has got the right abstractions for the problem you are trying to solve
  • prefer programmable tools

The last point is particularly important because it contributes to my maker side and liberates me from the (arguably understandable) "solve for the 80% of the use cases" mantra of many products out there.

What I am trying to improve

Like many others, my daily routine consists of having to deal with backlogs and tickets.

Our corporate tooling can correlate tickets with actual work through a very simple convention in the git commit messages:

[PM-1824] Rename config keys

This patch makes the config keys a bit shorter.

The part within brackets is the ticket number I am working on.

At the beginning of each week, each developer collects the ticket numbers and submits them to some other corporate tool.

This is currently a manual step and you tend to conveniently "forget" about it. Whenever the punctual reminder loudly knocks at your email the only thing you can do is to rush through your commits, checking all the different repositories you contributed to...trying to do the job that machines have been designed for.

Not fun.

Org-mode: the swiss army knife and requirements

In the spirit of the tool selection guidelines laid out above I have wholeheartedly embraced the way of the Emacs for a long time now.

Emacs is not an editor but ultimately a productivity tool. The productivity tool I would say.

It is infinitely programmable (via Emacs Lisp) but it has also been around for enough time that a myriad of plugins 1 have been developed for it. This allows you to reduce context switching: I do most of my coding, git 2 and even part of my web browsing in Emacs.

Not too bad for a piece of software initially released in 1976.

Is Emacs the right tool for my particular problem?

Absolutely, enter org-mode:

Org-mode [...] is a document editing, formatting, and organizing mode, designed for notes, planning, and authoring within the free software text editor Emacs.

Just by looking at the above description one can be quite confident that we are going to have all the pieces we need to improve our workflow. The whole of org-mode simply boils down to a thin layer of very well oiled conventions so that text can be interpreted as data.

Let's start with what we need then. We would like to have:

  • An easy way to store ticket numbers each day, potentially many times per day
  • An easy way to display weekly reports

Expanding on the first point - it does not need to be fully automated as long as it is fully integrated into the code & commit flow.

Org-capture: parsing and storing commit messages

The org-mode capture functionality is what allows you to store text and metadata interactively in a .org file of your choice. The text can be anything useful, like a note for your future self, an idea for a blog post, a snippet of code.

The only thing you tell Emacs is what metadata should be captured and the text that goes along with it.

With a bit of bottom-up approach, let's prepare the very first functions we will be using for parsing the commit message string so that ticket number and actual commit message can be isolated:

(defun org-conf--find-commit-ticket (text)
  "Find the ticket number from the input TEXT."
  (when (string-match "\\[\\(.*\\)]" text)
    (match-string 1 text)))

(defun org-conf--find-commit-msg (text)
  "Find the actual commit message from the input TEXT."
  (when (string-match "\\[.*\\]\\s-+\\(\\(.\\|\n\\)*\\)" text)
    (match-string 1 text)))

I hope that the only unfamiliar piece is the org-conf-- prefix 3 up there. The rest is simply how you do regex group matching in Emacs Lisp, plus an awful lot of escaping.

The way you hook this up in org-capture is to create a custom template:

(setq org-capture-templates
      `(("w" "Work Templates")

        ("wc" "Commit Ticket"
         entry
         (file+olp+datetree ,(concat org-directory "/agenda/tickets.org.gpg"))
         ,(string-join
           (list "* %(org-conf--find-commit-ticket (org-conf--retrieve-commit-text)) :dev:"
                 ":LOGBOOK:"
                 ":added: %T"
                 ":END:"
                 "%(org-conf--find-commit-msg (org-conf--retrieve-commit-text))%?")
           "\n")
         :clock-resume t
         :tree-type week)))

There is a log going on there but "w" and "wc" are the shortcuts (and description, more on this later) used in the org-capture interactive menu. Then one can easily detect the (encrypted in my case) tickets.org destination file and the :added: %T entry timestamp 4.

The first item of the list sexp needs more explanation:

The long-winded %(org-conf--find-commit-ticket (org-conf--retrieve-commit-text)) means, in lispy words, call org-conf--retrieve-commit-text and pass its result to org-conf--find-commit-ticket.

We have already seen the latter. The former is the new piece of custom code that retrieves the commit message. It grabs it either from Emacs' selection (called :initial in org-mode) or, if not there, from magit's very handy git-commit-buffer-message. It looks like this.

(defun org-conf--retrieve-commit-text ()
  "Return INITIAL or try to call git-commit-buffer-message."
  (cond
   ((let ((captured (org-capture-get :initial)))
      (when captured captured)))

   ((fboundp 'git-commit-buffer-message)
    (with-current-buffer (org-capture-get :original-buffer)
      (git-commit-buffer-message)))

   ((t nil))))

What ends up in ticket.org looks like this (some entries have been collapsed for clarity):

captured tickets example

As you can see, the ticket number is under a very specific heading, categorized by the week number. You get this for free because of the file+olp+datetree target directive above 5.

Org-agenda: reports and improvement achieved

The other side of the coin is the weekly report. This is even easier than the above - it is mostly configuration:

(setq org-agenda-custom-commands
      `(("c" "Weekly Commit Tickets"
         ((agenda "" ((org-agenda-files (list ,(concat org-directory "/agenda/tickets.org.gpg")))
                      (org-agenda-span 'week)
                      (org-agenda-start-on-weekday 1)
                      (org-agenda-overriding-header "Worked on tickets: ")
                      (org-agenda-time-grid nil)))))))

Every time I need to compute my ticket numbers for the current week, I type C-c a (the org-agenda Emacs function) and then c. The b and f shortcuts bring me to the previous and next week, respectively. Like the above, many other useful shortcuts have been tailored by decades of contributions for you to use.

The report is very simple but effective.

agenda weekly report

With these additions, at any point during my day I can C-c c (org-capture Emacs function) and an interactive menu pops up.

capture selection dialog

At that point, I can press w and c for adding an entry. The entry is displayed to me before appending so that I can tweak it or kill it (C-c C-k). Most of the times I just confirm the addition by C-c C-c and the entry will appear in ticket.org, as we saw above.

In case I am not in a git commit buffer, I simply make sure that my selection contains a suitable string to be parsed. As long as there is a [FOOBAR] in it I am good.

Conclusions

Winding it up, I hope this blog post sparked a bit of curiosity on a rather unconventional but low-cost solution to a productivity issue.

For more details, here are some links I found useful along the way:

Happy hacking!


  1. These are the AUR packages (only).

  2. Please consider donating to Jonas Bernoulli if you use his flawless magit here.

  3. Emacs Lisp does not have either packages nor modules so we scope functions by their names.

  4. The :LOGBOOK: and :END: strings are demarcating the metadata section. Remember, a .org file is simple text after all - you need delimiters.

  5. There are many other targets of course and you can even pass a custom function.