Issue Tracking and the Health of Open Source Software

I read something recently that suggested that the health of an open source project and its community could be largely assessed by reviewing the status of the bug tracker. I'm still trying to track down the citation for this remark. This basically says that vital active projects have regularly updated bugs that are clearly described and that bugs be easy to search and easy to submit.

I'm not sure that free software communities and projects can be so easily assessed or that conventional project management practices are the only meaningful way to judge a project's health. While we're at it, I don't know that it's terribly useful to focus too much attention or importance on project management. Having said that, the emergence of organizational structure is incredibly fascinating, and could probably tolerate more investigation.

As a starting point, I'd like to offer two conjectures:

  • First, that transparent issue tracking is a reasonably effective means of "customer service," or user support. If the bug tracking contains answers to questions that people encounter during use, and provide a way to resolve issues with the software that's productive and helps with support self-service. Obviously some users and groups of users are better at this than others.
  • Second, issue tracking is perhaps the best way to do bottom-up project/product management and planning in the open, particularly since these kinds or projects lack formal procedures and designated roles to do this kind of organizational work.

While the overriding goal of personal task management is to break things into the smallest manageable work units, the overriding goal of issue tracking systems is to track the most intellectually discrete issues within a single project through the development process. Thus, issue tracking systems have requirements that are either much less important in personal systems or actively counter-intuitive for other uses. They are:

  • Task assignment, so that specific issues can be assigned different team members. Ideally this gets a specific developer can "own" a specific portion of the project and actually be able to work and coordinate efforts on the project.
  • Task prioritization, so that less important or crucial issues get attention before "nice to have," items are addressed.
  • Issue comments and additional attached information, to track progress and support information sharing among teams, particularly over long periods of time with asynchronous elements.

While it's nice to be able to integrate tasks and notes (this is really the core of org-mode's strength) issue tracking systems need to be able to accommodate error output and discussion from a team on the best solution, as well as discussion about the ideal solution.

The truth is that a lot of projects don't do a very good job of using issue tracking systems, despite how necessary and important bug trackers. The prefabricated systems can be frustrating and difficult to use, and most of the minimalist systems [1] are hard to use in groups. [2] The first person to write a fully featured, lightweight, and easy to use issue tracking system will be incredibly successful. Feel free to submit a patch to this post, if you're aware of a viable systems along these lines.

[1]I'm thinking about using ikiwiki or org-mode to track issues, but ditz suffers from the same core problem.
[2]Basically, they either sacrifice structure or concurrency features or both. Less structured systems rely on a group of people to capture the same sort of information in a regular way (unlikely) or they capture less information, neither option is tenable. Without concurrency (because they store things in single flat files) people can't use them to manage collaboration, which make them awkward personal task tracking systems.

And Then I Broke Down and Got a Tablet

Ok, I caved and got a tablet. This is a post about my experiences with the tablet and some general thoughts on the format.

I opted for the Motorola Xoom. It's an Android device, I appreciate the Motarola build quality, and I'm very pleased with my choice. First impressions first:

  • Reading on the tablet is great. I have a Kindle, and while I respect how lightweight the Kindle is itself. Despite the extra weight, the slightly larger screen and the back light is very very nice and very welcome.
  • I don't expect that I'll be doing a lot of writing on the tablet, a laptop is never really going to be that far away, but I'm really surprised by how easy it is to (almost) touch type on the tablet. A number of very simple and probably straightforward innovations to the keyboard could make things so much better.
  • I think all devices need some sort of "don't auto rotate" hardware switch. In fact, I think apple's whole "lets get rid of hardware buttons," movement to be really annoying. Buttons should be overloaded, sure, but I hate having to hunt through menus to modify basic behavior. Having said that, the "software control bar" at the bottom of Android 3.0 is brilliant and a good move (given screen rotation.)
  • I lament not having a Google voice widget for the tablet. Makes sense that they wouldn't want this for tablets that had data plans, but I just have a wifi tablet.
  • The Kindle app doesn't let you bookmark your place in periodicals. Which might make sense if you were reading the Times, but doesn't make a lot of sense when reading fiction magazines with articles in the rage of 10k words.
  • I'm in love with the calendar application, except for the "full month view," in which you scroll by weeks, not by months. Even with this glitch, I'm curious as to why there aren't (stand alone) calendar applications of this quality for desktops.
  • I've tended to use the tablet for situations where I want to have a distraction free experience (usually for reading,) or where I want to do "computer things" in a situation where I might need to interact with other people. Having a tablet in your lap is more social than a laptop. As such, I don't think it would ever be able to replace a "real" computer for very long, but that doesn't make it less useful.

I'll be writing more about the tablet experience and some cyborg features of tablet use and usability.

Onward and Upward!

Interfaces in Enterprise Software

This post is a continuation of my human solution to IT and IT policy issues series. This post discusses a couple of ideas about "enterprise" software, and its importance the kind of overall analysis of technology that this posts (and others on this site) engage in. In many ways this is a different angle on some of the same questions addressed in my "Caring about Java" post: boring technologies are important, if not outright interesting.

There are two likely truths about software that make sense upon reflection, but are a bit weird when you think about it:

  1. The majority of software is used by a small minority of users. This includes software that's written for and used by other software developers, infrastructure, and the applications which are written for "internal use." This includes various database, CRM, administrative tools, and other portals and tools that enterprise uses.
  2. Beautiful and intuitive interfaces are only worth constructing if your software has a large prospective userbase or if you're writing software where a couple of competing products share a set of common features. Otherwise there's no real point to designing a really swanky user interface.

I'm pretty sure that these theories hold up pretty well, and are reasonably logical. The following conclusions are, I think, particularly interesting:

  • People, even non-technical users, adjust to really horrible user interfaces that are non-intuitive all the time.

  • We think that graphical user interfaces are required for technological intelligibility, while the people who design software use GUIs as minimally as possible, and for the vast majority of software the user interface is the weakest point.

    The obvious questions then, is: why don't we trust non-technical users with command lines? Thoughts?

Create Better Task Items

I was paging through a list of things that I made for myself during a call I was in a few weeks ago, and was utterly dismayed by how useless the items were on the list. I wasn't sure what needed to be done, I couldn't remember what things meant, and I was left with the sinking suspicion that I had forgotten something crucial. I write this, in part, as a lesson to my past self on how to write good task list items.

Hopefully you'll find it useful.

Task items must be actionable. You need to be able to read the subject or summary and know: what the project is, what kind of work it is, what needs to be done, and what very next thing you need to do is.

Tasks cannot be open ended. It's really tempting to write tasks in the form of "work on a project" or "make progress on email backlog," but don't. How do you know if you've done the task? Is all progress the same? Is the actual work activity plainly obvious from an open ended task description?

Tasks need to concise. I'm a big fan of including some sort of status information and some sort of instruction and context with your tasks, but you need to be able to look at a task list and triage what to do next without thinking very much and without spending more than a few seconds deciphering messages from your past self. Write good summaries.

Try to organize your projects and tasks so that most of your task items are not dependent upon other items. Sometime dependencies are unavoidable, but I find if you're clever, you can chop things up into parallel tasks that are easier to work on but that accomplish the same goal. In some cases, long strings of dependent tasks can be just as troublesome as large open tasks, because in the moment they amount to clutter.

Also your feedback and suggestions from your own experience may be of interest to all of us! I look forward to hearing from you!

Mobile Productivity Challenges

I've never really figured out how to do work with anything less than a full computer. I've tried everything: Palm Pilots and Pocket PCs in the early 2000s, laptops, and eventually I just settled on just dragging a (smaller) laptop almost everywhere circa 2005. I get the feeling that, most people who have been thinking about mobile technology and productivity assume that the only impediments to mobile productivity are better hardware and software. Contemporary (multi)touch screen devices are the current embodiment of this theory.

I'm convinced that the theory is wrong.

Having better and more powerful technology doesn't hurt anything. I'm a huge fan of the smaller, integrated, and more powerful devices. Software written specifically with mobile users in mind does improve the potential for productivity. These technological improvements, however, make the underlying problem more apparent.

The challenge of getting things done when mobile has little to do with the capabilities of the mobile platform, and more with the way people think about and plan work when mobile. Not only is this hugely frustrating to users, but technological capability that people can't use threatens the ongoing development and adoption of new technology.

Using Mobile Technology More Effectively

The solution, here, I think is two-fold:

Fully Integrated Applications

Let's develop integrated applications, not just integrated devices with different applications. Just as we didn't need separate devices for every mobile function: telephony, music playing, book reading, mobile internet, and so forth. We don't need different applications for every function: calendaring, messaging, email, contact management, notes, reading, and so forth.

At the very least applications need to be highly interoperable, so that users can send data between application functions easily, and synchronize data back to desktop and web portals seamlessly.

Task Planning Strategies for Mobile Productivity

I don't think that the "user stories" for mobile technology are really fully developed, and as a result any interaction with a mobile device that isn't responsive (i.e. there's an alert of a new event, and people respond to it,) is either "twiddling nobs" (i.e. non productive,) or entertainment focused (i.e. playing music, video, or opening a book.) Perhaps that's enough for some uses, but these this kind of workflow covers a small percentage of what people do with computers.

If mobile technology is going to replace a general purpose laptop, ever, even in limited situations, we need to figure out how to work in different ways. I know that I am loosing a great deal of time, when I'm using my phone switching between the notes app, the reader, the task list, and the calender. This task switching gets in the way of doing things to a much larger extent than similar behavior does when using a conventional computer. I would even posit that, the cost of context switching is inversely related to the size of the interface.

Better application integration will help this, but I think the real solution is providing a method for people to organize their mobile time more effectively. The task list that we build for ourselves when we're doing "conventional" work (i.e. things that we need to remember to do, open projects, open issues,) aren't particularly useful or usable when we're looking at a tablet or a phone. If we don't know what we ought to be doing, it doesn't matter what the device or software is capable of in theory.

There are probably a dozen or more solutions to this problem, but here's my first stab at it. What if there was a way to "forward tasks" to ourselves when we're on the run, but have a few moments? We all loose time waiting in queues, or waiting for trains, and these seem like ideal phone times. If we had a way to queue things for ourselves, so we could spend the time doing something. Even better, would be software that would not only collect and display the queue but would also connect with the application where whatever needed to be done was and then record the results and send the back to our desktops when we were done.

Thoughts?

Better Task Lists

Just about everyone keeps a task list of some sort, or has at some point. To the casual observer, task list management might seems like a simple problem that could be augmented with a little bit of automation for great effect. Fire up your nearest "app store" and I would bet money that you'll find a at least a few developers that have had the same thought.

For such a seemingly simple engineering problem there is an inordinate amount of really bad software. While this might tempt us to reassess the complexity of the task management problem, I don't think this is really true. What happens, I'm convinced, is that people (i.e. cyborgs) make lists of tasks to solve different problems in their realities, and these different lists often require different automation. So while there are 10-20 basic task management applications, the number of distinct usage profiles exceeds that by several times. That's the theory at any rate.

Archetypes

Allowing for large amount of diversity, there are still a few generally useful task list "archetypes" that we can use to characterize how people use task lists. I just want to enumerate them, here for now. I might move them out to another page, and you should feel free to edit the page (it's a wiki!) if you think I've missed anything!

  • Collaboration Facilitation: Teams need systems to keep track of and work on shared issue queues. These are "bug trackers," and are totally essential when items or "issues:" take a significant time to complete, require the effort/input/awareness of multiple people, and need to be created or added by a number of people.
  • Memory Enhancement: People create todo lists when they're working more things than they can comfortably remember at any one time. The lists tend to be ephemeral, the items can be quickly resolved, but we make these lists so we don't have to remember a long list of things while working. Think post-it notes.
  • Obligation Management: These lists bring us close to calendars, but we keep them to make sure we remember to do required tasks. Often these lists are helpful in helping people make sure that they're "caught up," so that they can enjoy and use free time without interruption or nagging.
  • Task Prioritization: When time is limited, a list of tasks is useful in organizing an order, and making use of available time, so that it's possible to keep track of all open tasks, while also being able to allocate effort and time to tasks in a smart way that accounts for available time, importance, and deadlines. The goal is to get the most crucial things done while also never wondering what one could be doing with a few free moments.
  • Progress Tracking: These lists are less to track things todo and more to track have done. When working on a number of long term and short term projects, a list of what's open, what's been finished, along with a status of where things are is useful to avoid loosing track of projects and tasks.

Based on this, I think we can distill a number of overriding qualities of task lists from these five archetypal use cases. That working list is:

  • item granularity,
  • project-level organization,
  • scheduling and deadlines, and
  • use of priority markers. ## Personal Case Studies

When I had an epic commute my issue was that while I had free time, it was all in 20 or 40 minute segments on trains. The challenge was to be organized and focused enough to really use this time. It was early and while I was awake enough to get work done, I wasn't always awake enough to figure out what needed my attention. And I didn't have enough routine minutia or enough other free time to be able to spend these blocks of time doing minutia. I needed a task list that told me what to do and when to do it. I needed Task prioritization with a little obligation management or something close to that. I chopped every task into the smallest actionable items, which is annoying in creative projects and often not very useful, and then scheduled items out so that I could do 6 or so things a day, and anytime I opened the laptop there was a list of things I could jump into. It worked, more or less.

Now, I have free time. I even have a few hours strung together. The issue is less that I need help filling in every little moment with something to do, and more that I have too much that I could be working on, and I need help figuring out what the status is of ongoing projects are and where I ought to things are and what needs my attention the most. Progress tracking, more or less with a little task prioritization but in a very different way than I'd been doing. it. I've got a lot of things, and I need to be able to see where projects, and what needs attention now. I've not figured out the best solution, but I think less scheduling and bigger conceptual task objects is more the way to go.

Does this way of thinking about things make sense to other people?

Moving the Furinture

A few weeks ago we moved the furniture around. The biggest objective was to get some desk space setup in a more usable fashion. I find myself doing this a couple times a year. It's easy for me to get in the habit of having everything a certain way without realizing that the chair I like to sit on is always in the sun during the mornings when I like to write, or that I never put the dishes away, because the cabinets were arranged backwards. The same sort of thing happens with computers and the way we use technology.

Matt Lundin and I were talking about this a couple of weeks ago. I think at the core of the issue is that the kind of work we do with computers, changes subtly: either what we do actually changes as a result of our progress, or we learn more about what we're doing and what we ought to be doing from completing work. Like the furniture, it's easy to get accustomed to doing things in one particular way and not realize that there's a better way.

It's useful then to spend a little time every now and then to do a little bit of digital spring cleaning. There are two major parts of this process:

3. Figure out if "what you do" has changed significantly. As our projects grow the work we do changes, sometimes in ways that make it hard to continue to organize our efforts and tools in the same way.

4. Looking for new tools that you can use to do what you do. This includes flat out new tools of which you weren't aware, little automations that you may be able to build, and tools that solve problem domains that you'd previously ignored. It's also sometimes useful to look at your tools and figure out what is more trouble than it's worth. Ideally these don't all change annually, but it's worth doing a review.

There's no guarantee, of course, that you'll find a solution to any issue that you come across but you might. You might also be able to gain new insights into issues that have nagged at you by approaching a number of issues and pain points all at once. It's also, I'm pretty sure, more productive to spend most of your time doing stuff (writing, coding, etc.) even if your world isn't perfectly optimized, than it is to spend all your time tinkering with things.

And the sofa really does look better over there.

Onward and Upward!

Bad Org-Mode Habits

Org-mode is great. Org is this super-task management package that merges outlining and structured document editing abilities with task management glue. It sounds like a weird combination at first, when you realize that it means that you can effectively do work and organize your work in the same set of files without needing to "switch modes," between a planning/task list interface and a "doing things" interface, the effects are amazing.

I'll leave "the awesomeness org-mode" to other people and other posts, and spend time here focusing on why "org-mode is awesome but won't do your work for you," and "if you're new to org-mode you will probably want to do things in a certain way, but don't" issues. Org-mode is great and it can be even better if you avoid developing a few bad habits.

Use `org-capture <http://orgmode.org/manual/Capture.html>`_ as much as possible. Org-capture is a quick interface within emacs that lets you open a temporary buffer, take a few notes, and save the notes into a file. It has advanced features for more complex data entry features and data types. The temptation is to use it as a "quick entry" tool for task list items, but don't just use it to capture new tasks. Capture links and bookmarks, store notes and important information, If org-mode is your outboard emacs-based brain, then org-capture is it's main input device. Fighting it, means that your org files end up being less useful.

Avoid using org-mode as a simple task list, and particularly avoid constructing the content in your org files to "game" your agenda view. The agenda compiles a working task list for your actionable notes, working backwards from this means your notes are less than useful things can get lost and all of the really cool things that org lets you do aren't accessible to you.

This may just be the application of a good general information management practice, but: distribute information evenly throughout all levels of the organizational hierarchy of the file. Use the org-narrow-to-subtree function to focus your current work on a specific portion of a file, and don't bury information or have it sitting around in one big unorganized pile.

Hope this helps!