Issue Tracking and the Health of Open Source Software

I read something recently that suggested that the health of an open source project and its community could be largely assessed by reviewing the status of the bug tracker. I'm still trying to track down the citation for this remark. This basically says that vital active projects have regularly updated bugs that are clearly described and that bugs be easy to search and easy to submit.

I'm not sure that free software communities and projects can be so easily assessed or that conventional project management practices are the only meaningful way to judge a project's health. While we're at it, I don't know that it's terribly useful to focus too much attention or importance on project management. Having said that, the emergence of organizational structure is incredibly fascinating, and could probably tolerate more investigation.

As a starting point, I'd like to offer two conjectures:

  • First, that transparent issue tracking is a reasonably effective means of "customer service," or user support. If the bug tracking contains answers to questions that people encounter during use, and provide a way to resolve issues with the software that's productive and helps with support self-service. Obviously some users and groups of users are better at this than others.
  • Second, issue tracking is perhaps the best way to do bottom-up project/product management and planning in the open, particularly since these kinds or projects lack formal procedures and designated roles to do this kind of organizational work.

While the overriding goal of personal task management is to break things into the smallest manageable work units, the overriding goal of issue tracking systems is to track the most intellectually discrete issues within a single project through the development process. Thus, issue tracking systems have requirements that are either much less important in personal systems or actively counter-intuitive for other uses. They are:

  • Task assignment, so that specific issues can be assigned different team members. Ideally this gets a specific developer can "own" a specific portion of the project and actually be able to work and coordinate efforts on the project.
  • Task prioritization, so that less important or crucial issues get attention before "nice to have," items are addressed.
  • Issue comments and additional attached information, to track progress and support information sharing among teams, particularly over long periods of time with asynchronous elements.

While it's nice to be able to integrate tasks and notes (this is really the core of org-mode's strength) issue tracking systems need to be able to accommodate error output and discussion from a team on the best solution, as well as discussion about the ideal solution.

The truth is that a lot of projects don't do a very good job of using issue tracking systems, despite how necessary and important bug trackers. The prefabricated systems can be frustrating and difficult to use, and most of the minimalist systems [1] are hard to use in groups. [2] The first person to write a fully featured, lightweight, and easy to use issue tracking system will be incredibly successful. Feel free to submit a patch to this post, if you're aware of a viable systems along these lines.

[1]I'm thinking about using ikiwiki or org-mode to track issues, but ditz suffers from the same core problem.
[2]Basically, they either sacrifice structure or concurrency features or both. Less structured systems rely on a group of people to capture the same sort of information in a regular way (unlikely) or they capture less information, neither option is tenable. Without concurrency (because they store things in single flat files) people can't use them to manage collaboration, which make them awkward personal task tracking systems.

Advice for Blogging Successfully

Although I forget it, sometimes, the following video is probably the single best piece of advice for better blogging. Watch the video:

Cory Docotorow - How to be an Uber Blogger

Attention and time are scarce while content is plentiful. "If you write it they'll read" cannot, thus, be true of blogging. Interesting and important content is interesting, but success lies in managing the attention economy and focusing on output that is easy to read and easy to skim. Its not glamorous, and it requires giving up a bunch of pride, but writers must write with readers in mind.

I'll skip the meandering analysis and get to a couple of key questions that I think remain open, even 4 years after this video was posted:

  • What about non-newsreel blogs? Blogs that are more analysis and less regurgitation's of boingboing/metafilter/slashdot?
  • What about non-blog content? Are books and articles, subject to the same overload (yes?) but are the solutions always "write easy to process, easy to skip," content? (Maybe?)
  • The connection between the "attention" economy, evolving search engine use patterns, and mobile technology change the way that we interact with and compensate for overload?

Discuss!

Why Cloud Culture Isn't

I read this post by Bruce Sterling about "Cloud Culture", and haven't quite been able to formulate an appropriate response. It's true that I often find myself ranting about this or that thing Bruce Sterling has written, so I'm perhaps not the most impartial

To summarize, his notion is that "the cloud," as evidenced by a wealth of usable digital information and content, combined with an uptick in digital participation will lead to some sort of minimalistic cultural singularity that he's calling "Cloud Culture."

Right.

So lets attack this full on. The "Cloud" is marketing dribble. There's nothing there, it's just a different way of organizing the servers that provide content and services on the Internet. In a lot of way the Amazon and Rackspace "Clouds" have brute forced the server and hosting business with utterly conventional hardware, off the shelf open source software, and a bunch of [insert-favorite-scripting-language-here] scripts. Oh, and a lot of capital.

That's the back-end, and moving upwards in the stack, we'll find that the tools (e.g. Apache, Relational Databases, Perl) that we're using now are for the most part the same this time around as they were ten or fifteen years ago. Sure we have new practices, and there are new frameworks, new versions of some software, but the changes are mostly evolutionary. Until you get to what the users see, and now we have applications that live in web browsers, and a more robust set of interconnected network services.

Otherwise, the cloud is just the Internet bundled up in a differed way. And it's true that the bundling matters, but at the end of the day just because the "cloud" presents a better way to do the work of the Internet (in some situations,) it doesn't mean that any of the fundamental principals have changed.

Another phenomena that hasn't changed with the cloud: we've not quite gotten the economic system for the Internet figured out in a way that's reliable and sustainable. New things on the Internet are supported by capital from "old-world" industries by way of venture capital and spin off subsidiaries. The leading business models are advertising (very print media) and premium subscription services (which are often contrived). So eventually it all collapses, and any gains in cultural participation run up against the hard costs of keeping power flowing to the servers in some way.

A few thoughts:

  • We could just as easily call this whole "cloud" thing "the matrix," or "cyberspace," in Gibson's original formulation. Cloud is a horrible metaphor for most things, but particularly this.
  • The apparent cohesiveness of the Internet in 2010 is almost entirely due to the success of Google. As much as we might try and avoid Google, I firmly believe if Google were to go away without warning, while the Internet would still work most of the economy around it would crash and burn: people wouldn't discover new things, the on-line advertising industry would largely collapse, a lot of the software that a lot of people inside and outside of google use would be without maintainers.
  • I'm not sure cultural participation is problem space that can be brute forced, and certainly the development of additional applications in the cloud is unlikely to increase participation in a direct manner. I'm not being an elitist and saying "cultural participation by the unschooled and unwashed masses isn't worthwhile," but rather cultural participation is about developing direct relationships and making/building things, and that requires forming relationships that don't scale well.

Thoughts?

If Open Source is Big Business Then Whither the Community?

I've been thinking recently about the relationship and dynamic between the corporations and "enterprises" which participate in and reap benefits from open source/free software and the quasi-mythic "communities" that are responsible for the creation and maintenance of the software. Additionally this post may be considered part of my ongoing series on cooperative economics.

When people, ranging from business types, to IT professionals, to programmers, and beyond, talk about open source software we talk about a community: often small to medium sized groups of people who all contribute small amounts of time to creating software. And we're not just talking about dinky little scripts that make publishing blogs easier (or some such), we're talking about a massive amount of software: entire operating systems, widely used implementations of nearly all relevant programing languages, and so forth. On some level the core of this question is who are these people, and how do they produce software?

On the surface the answer to these questions is straightforward. The people who work on open source software are professional programmers, students, geeks, and hacker/tinkerer-types who need their computers to do something novel, and then they write software. This works as model for thinking about who participates in open source, if we assume that the reason why people contribute to open source projects is because their individual works/contributions are too small to develop business models around. This might explain some portion of open source contributions, but it feels incomplete to me.

There are a number of software projects that use open source/free software licenses, with accessible source code, supported by "communities," which are nonetheless developed almost entirely by single companies. MySQL, Alfresco, and Resin among others serve as examples these kinds of projects which are open source by many any definitions and yet don't particularly strike me as "community," projects. Is the fact that this software provides source code meaningful or important?

Other questions...

1. If there are companies making money from open source code bases, particularly big companies in a business directly related to software, does this effect participation of people who are not employed by that company in the project?

In my mind I draw distinctions between technology businesses that use/sell/support open source software (e.g. Red-Hat, the late MySQL AB, etc.) and businesses that do something else but use open source software (i.e. everyone with a Linux server in the basement, every business with a website that runs on Apache, etc.)

2. Does corporate personhood extend to the open source community. Are corporate developers contributing as people, or as representatives of their company?

I largely expect that it's the former; however, I'd be interested in learning more about the various factors which affect the way these contributors are perceived?

3. Do people participate in open source because it is fun or for the enjoyment of programming,

4. Has software become so generic that open source is a current evolution of industry standards groups. Do we write open source software for the same reason that industries standardized the size and threading of bolts?

5. Are potential contributors disinclined to contribute to software that is controlled by a single entity, or where projects

6. Is the cost of forking a software project too high to make that a realistic outcome of releasing open source software?

Conversely, were forks ever effective?

7. Do communities actually form around software targeted at "enterprise" users, and if so in what ways are those communities different from the communities that form around niche window p managers or even community projects like Debian?

I don't of course have answers yet, but I think these questions are important, and I'd love to hear if you have any ideas about finding answers to these questions, or additional related questions that I've missed.

Blogging Forms

One of the aspects of "BloggingFail" [1] during the most recent technology/new media bubble, is the emergence of "blog post formulas," which are basic post formats that people use to provide structure to a post, and produce content in a way that's more readable for casual visitors, and better for search engines.

Interestingly, not all of these formats are as bad as the BlogFail that they helped create, and I've been interested in collecting/creating a few new and different formats for blog posts. This is a collection of those post templates.

The "N Things Post"

This is big, and very mainstream we see this a lot as filler, and I think it grows out of the kinds of articles you see on news stands. Basically the gimmick (and I think it works) is that it promises a post that will be easy to read, provide information in clear ways, and won't encumber a collection of information with complicated rhetoric.

It works, there was a long time when digg was filled with "N Things posts" of dubious merit. Having said that, it's a great format for presenting some kinds of information.

Tip: While we're at it, it helps if the N is a prime number (eg. 5, 7, or 11 make good Ns for N Things Posts)

The RedMonk Interview

This is mostly Stephen O'Grady's invention, though I've seen it elsewhere. The basic idea is that you ask yourself a bunch of questions and then answer them yourself.

While this might sound contrite by my description, the posts that result are often quite successful at communicating information. In a sort of not-very-subtle way, you're able to frame your discussion by communicating to your reader what questions you think are the most important. It again wins by stripping away potentially complex and linear rhetoric, and lets you sort of jumpstart the conversation that inevitably follows.

The Synthetic Review

I've started doing this more, and it's a form I've totally yanked from Academic journals and other similar sorts of outlets. Basically, you take two or three articles--potentially related, sometimes not--put links to them at the top of the post and then respond to each post and to the juxtaposition created by putting the links together in one post.

The questions I (try to) ask myself are both "what do I think of each of these articles," and "what would these articles say about each other."

The Pattern/Tutorial

This is the "here's how to do something" post. We see this in a lot of genres, from knitting, to technology, to cooking and back again. These posts tend to be both extremely popular and successful, but they are also quite useful to readers, new and old.

When I got into blogging, we were much closer to the "journal" end of the spectrum (as a community), but I think the transition to being about providing/creating value is something that's really emerged in blogging in the last ten years, in part because of the prevalence of a class of posts like this. So there you have it.

[1]This is, in my estimation, what happened as a result of the hype around "new media," "social media," and "search engine optimization," that resulted in an explosion in the number of blogs between 2006 and 2009. Blogs which are mostly designed to generate advertising revenue, rather than stimulate useful conversation. This isn't to say that there's nothing good out there, but I think we've all come across blogs that fall into this category, and it's always apparent. In my weaker moments, I call it the ProBlogger phenomena.

The Blog is Dead, Long Live the (micro)Blog

"I'm giving up blogging because twitter has more energy and satisfies my online media needs these days." I here yet another person say, as they give up the blog that they've been working on sporadically for the last 4 or five years for a twitter account.

I'm certainly not giving up blogging any time soon, but I hear people say these things. Not always so explicitly, and less often now that twitter has become more established, and less of a novelty. Nevertheless I think its high time to take a step back and take an account of "the state of blogging."

While I think we need to consider the impact of twitter on the current state of blogging, I think the past five years and maybe the past seven or eight years (most of which have been without twitter) have had an even larger impact on the forum.

I'm not sure, exactly, what the state of things are, but the following are the questions I'm asking myself.

  • Are blogs simply the default way of publishing serialized/periodical content and updates to websites?
  • Blogging, at least in my mind, grew out of online-journal communities, and while there's a lot of division between "bloggers" and "journalers" there's a lot of connection. Blogs can be self-referential, and first-person, and they can drift between multiple threads of the author(s) life. What's the state of blogging/journaling?
  • Are blogs things that people grow, develop, and build over a long time, or are blogs commodities that serve a specific purpose attached to some other purpose. In other words, do people say, I want to create a blog, and they have a blog which meanders and continues for years, or have blogs become something that people start on a whim in response to communities or current events, and then discard when the mood passes?
  • Do people read blogs? I have a good excuse for being more than a thousand post behind on my feed reader (moving across the country, starting a new job) but I'm pretty sure that blog reading isn't exactly flourishing. There are some really well read blogs, of course, but I don't know if people are really reading.

My answers, if not obvious are: yes, strained and under-appreciated, more commodity and ephemeral than they used to be as a result of software development, and readership hasn't grown with the growth of the web.

And then we introduce twitter.

I've always seen twitter as an evolution of the "chat room" of "IRC" and phenomena like that, rather than an evolution of the blog, though it makes sense to think about twitter and related formats as being "microbiology." At the same time, I think microblogging becomes a viable format because it makes it "OK" for folks to post lots of little ephemeral thoughts, which is hard in conventional blogging, both in terms of time/energy, but also in terms of what the software and social convention will allow.

In order for a blog post--just one--to be "successful," in today's world, it needs to be clever and well written, and it needs to hang around for long enough for people to notice it. It might also need to provide a useful analysis in combination with some useful information.

In order for a post to twitter to be successful, it needs to be and timely (so that people see it), it probably needs to include some sort of link, and other people need to "Retweet" it a lot (which has got to be the most annoying thing in the short history of the medium).

I don't think the "short form" is going to kill the long form, or that that has even begun to happen, but might twitter kill off some of the cruft that that's built up around commodity blogging? Does twitter reintegrate the journal-form with the more-objective form?

Maybe. We'll see in a little while.

The State of the Discourse

This isn't a fully formed thought yet, but I was wondering what the status of discussion and commenting is on the web these days. Clearly microblogging like Twitter and Identi.ca produces a powerful platform for conversations and I think what's coming with xmpp (innovative interfaces for group chats, etc.) furthers the potential for conversation online. At the same time, I'm wondering what the status of conversations are "older" media like blog comments...

Are people still commenting on blogs? A few of you comment here now and then, and websites like making light have vibrant comments threads (that I don't have the attention span for or time to read), and the big sites (slashdot, digg, etc.) have active comments as well, but a lot of sites (including those with moderate readership) don't get many comments, and my sense is that a significant percentage of comments these days are in the "me too" vein, rather than productive themselves (because threads are difficult to read). Here are a few questions:

  • Do features (threading? email notification? persistent identities?) make commenting "work better" or flow more productively?
  • Are conversations about content moving away from comments into more centralized media like twitter, email lists, and discussion forums?
  • Are we more likely to respond to a blog post we read in our own blog, rather than in a comments thread? Has the blogging community reached a saturation point?
  • Dose a vibrant community of comment-posers indicate a marker of blog-success, these days? Did it ever? What might replace comments?

I'm pretty convinced that comments are dying, but I'd love to get your feedback. I'm not terribly mournful about this but I'm very interested in thinking about how we (as a community will replace this niche.)

Microblogging Aggregation

When Jaiku gave microbloggers (ie users of services like identi.ca twitter, etc.) the ability to aggregate their content onto Jaiku from blogs, flickr, delicious, and of course twitter as well, I noted that it wouldn't take much for the Jaiku to become a ghost town, where no one would have to actually create new content or read in order for new content to filter through the site. While syndication allows people to do some really amazing things with content, without filtering, context, and moderation it's really easy to become overwhelmed. Frankly Jaiku provide(s/ed) enough filtering ability to make this useful but this "problem" continues:

There are services like ping.fm which let you type once and send widely to a whole host of audiences on a bunch of different websites. Which is great: it lets you be present without taking up all your time, it allows you to reach a very fractured audience. Good stuff. Right?

Except not so much. The thing about "social media" and these new very conversational media is that they're not so much about sending widely. They're great for reaching people, in the moment, but they're really more about having conversations with a community.

So we need something better. Because being shut off from your friends because you use different websites is bad; but what amounts to using AM radio cranked up loud to talk to your friends in a couple of different coffee houses is worse. I think open micro blogging (like identi.ca and laconica) is a step in the right direction, better independent profiles and curatorial tools would be another.

Thoughts?