Like issue tracking systems, documentation publication systems are never quite perfect. There are dozens of options, and most of them are horrible and difficult to use for one reason or another. Rather than outline why these systems are less than ideal, I want to provide a list of basic requirements that I think every documentation publishing system¹ should have.

Requirements

• Tag System. You have to be able to identify and link different pieces of content together in unique and potentially dynamic ways across a number of dimensions. Tagging systems, particularly those that can access and create lists of “other posts with similar tags,” are essential for providing some much needed organization to projects that are probably quite complex. Tagging systems should provide some way of supporting multiple tag namespaces. Also operations affecting tags need to be really efficient, from the users and software’s perspective, or else it won’t work at realistic scales.
• Static Generation. Content needs to be statically generated. There are so many good reasons to have static repositories. It allows you to plan releases (which is good if you need to coordinate documentation releases with software releases) most documentation changes infrequently. The truth is this feature alone isn’t so important, but static generation makes the next several features possible or easier.
• Development Builds. As you work on documentation, it’s important to be able to see what the entire resource will look like when published. This is a mass-preview mode, if you will. The issue here, is that unlike some kinds of web-based publications, documentation often needs to be updated in batches, and it’s useful to be able to see those changes all at once because the can all interact in peculiar ways. These test builds need to be possible locally, so that work isn’t dependent on a network connection, or shared infrastructure.
• Verification and Testing. While building “self-testing” documentation is really quite difficult (see also /technical-writing/dexy,) I think publication systems should be able to do “run tests” against documents and provide reports, even if the tests are just to make sure that new of software versions haven’t been released, or that links still work. It’s probably also a good idea to be able to verify that certain conventions are followed in terms of formatting: trailing white space, optional link formats, tagging conventions, required metadata, and so forth.
• Iteration Support. Documents need to be released and revised on various schedules as new versions and products are developed. Compounding this problem, old documentation (sometimes,) needs to hang around for backwards compatibility and legacy support. Document systems need to have flexible ways to tag documents as out of date, or establish pointers that say “the new version of this document is located here.” It’s possible to build this off of the tag system, but it’s probably better for it to be a separate piece of data.
• Version Control. These systems are great for storing content, facilitating easy collaboration, and supporting parallel work. Diffs are a great way to provide feedback for writers, and having history is useful for being able to recreate and trace your past thinking when you have to revisit a document or decision weeks and months later.
• Lightweight Markup. It’s dumb to make people write pure XML in pretty much every case. With rst, markdown, and pandoc the like there’s no reason to write XML. Ever. of story End.
• Renaming and Reorganization. As document repositories grow and develop, it seems inevitable that the initial sketch of the organization for the body of work change. Documents will need to be moved, URLs will need to be redirected or rewritten, and links will need to be updated. The software needs to support this directly.
• Workflow Support. Documentation systems need to be able to facilitate editorial workflows and reviews. This should grow out of some combination of a private tag name space and a reporting feature for contributions, which can generate lists of pages to help groups distribute labor and effort.

This might just be a quirk of my approach, but I tend approach documentation, terms of process and tooling, as if it were programming and writing software. They aren’t identical tasks, of course, but there are a lot of functional similarities And definitely enough to take advantage of the tooling and advances (i.e. make, git, etc.) that programmers have been able to build for themselves. Am I missing something or totally off base?

This post is adapted from a post I made to the org-mode email list a few weeks ago. I proposed an application to compliment MobileOrg for writing. Where MobileOrg collects the core bits of org-mode’s task planning functionality in a form that makes sense for smart phone users, the parts of org-mode functionality that people use to for writing and organizing the content of larger form projects isn’t particularly accessible.

I spend (or should spend) 70% or more of my time in front of a computer writing or editing something in org-mode. Most of my org files have tens of thousands of words of blog posts, notes, drafts of articles, and so forth. While I can store that data on an android device with only minor problems using a little script that I put together, and I can capture content into my org-files using email and some nifty filters, and there are text editors that can let me edit these files: it could be better.

The proposal is simple. Can we build something like Epistle for org-mode? It might just render org-mode text to HTML, and frankly that would be enough for me. If the editing interface had an org-indent-mode equivalent, org-syntax highlighting, and even collapsing trees or org-narrow-to-subtree, that’d be kind of like heaven.

I’m not a mobile developer, so I can’t promise to start making an app this instant if there’s interest but if anyone’s bored and thinks this might be a good idea (or knows of something that might work better for this.) I’d love to hear about it. If someone wants to start work on this, I’ll do whatever I can to help make this a reality.

Onward and Upward!

I’ve been using FBReaderJ to read .epub files on my tablet recently, and I discovered a nitfty feature: you can adjust the screen’s brightness by dragging your finger up or down the left side of the screen. Immediately this felt like discovering a new keybinding or a new function in emacs that I’d been wishing for a while time. Why, I thought, aren’t there more tricks like this?

The iPhone (and the iPad by extension) as well as Android make two major advances over previous iterations of mobile technology. First, they’re robust enough to run “real” programs written in conventional programming environment. Better development tools make for better applications and more eager developers (which also makes for better applications.) Second, the interfaces are designed to be used with fingers rather than stylus (thanks to capacitive touch screens) and the design aesthetic generally reflects minimalist values and simplicity. The mobile applications of today’s app stores would not work if they were visually complex and had multi-tiered menus, and hard to activate buttons.

The tension between these two features in these platforms makes it difficult to slip nifty features into applications. Furthermore, th economy of application market places does not create incentives for developers to build tools with enduring functionality. The .epub reader I mentioned above is actually free software.¹ I write a couple of posts a while back on innovation (one and two) that address the relationship between free software and technological development but that’s beside the point.

Given this, there are two major directions that I see tablet interfaces moving toward:

1. Tablet interfaces will slowly begin to acquire a more complete gestural shorthand and cross-app vocabulary that will allow us to become more effective users of this technology. Things like Sywpe are part of this, but I think there are more.

2. There will be general purpose systems for tablets that partially or wholly expect a keyboard, and then some sort of key-command system will emerge. This follows from my thoughts in the “Is Android the Future of Linux?” post.

I fully expect that both lines of development can expand in parallel.

I read something recently that suggested that the health of an open source project and its community could be largely assessed by reviewing the status of the bug tracker. I’m still trying to track down the citation for this remark. This basically says that vital active projects have regularly updated bugs that are clearly described and that bugs be easy to search and easy to submit.

I’m not sure that free software communities and projects can be so easily assessed or that conventional project management practices are the only meaningful way to judge a project’s health. While we’re at it, I don’t know that it’s terribly useful to focus too much attention or importance on project management. Having said that, the emergence of organizational structure is incredibly fascinating, and could probably tolerate more investigation.

As a starting point, I’d like to offer two conjectures:

• First, that transparent issue tracking is a reasonably effective means of “customer service,” or user support. If the bug tracking contains answers to questions that people encounter during use, and provide a way to resolve issues with the software that’s productive and helps with support self-service. Obviously some users and groups of users are better at this than others.
• Second, issue tracking is perhaps the best way to do bottom-up project/product management and planning in the open, particularly since these kinds or projects lack formal procedures and designated roles to do this kind of organizational work.

While the overriding goal of personal task management is to break things into the smallest manageable work units, the overriding goal of issue tracking systems is to track the most intellectually discrete issues within a single project through the development process. Thus, issue tracking systems have requirements that are either much less important in personal systems or actively counter-intuitive for other uses. They are:

• Task assignment, so that specific issues can be assigned different team members. Ideally this gets a specific developer can “own” a specific portion of the project and actually be able to work and coordinate efforts on the project.
• Task prioritization, so that less important or crucial issues get attention before “nice to have,” items are addressed.
• Issue comments and additional attached information, to track progress and support information sharing among teams, particularly over long periods of time with asynchronous elements.

While it’s nice to be able to integrate tasks and notes (this is really the core of org-mode’s strength) issue tracking systems need to be able to accommodate error output and discussion from a team on the best solution, as well as discussion about the ideal solution.

The truth is that a lot of projects don’t do a very good job of using issue tracking systems, despite how necessary and important bug trackers. The prefabricated systems can be frustrating and difficult to use, and most of the minimalist systems¹ are hard to use in groups.² The first person to write a fully featured, lightweight, and easy to use issue tracking system will be incredibly successful. Feel free to submit a patch to this post, if you’re aware of a viable systems along these lines.

By now, several weeks ago, in correspondence Matt Lundin that he thought Android was probably future of Linux," mostly as a throw away line. This feels like a really bold statement,¹ and I’ve enjoyed thinking about Android and “the future of Linux."²

On the face of it, Android is the future of Linux. Android is the Linux that most people will interact with before all others in a concrete manner. In all likelihood The future of Linux is probably mostly in running web servers, virtualization hosts, and any other server that matters. At this point, Linux’s platform support and use cases is far less interesting than its prevalence: the ubiquity of Linux, GNU, and BusyBox, is more import an that the fact that Linux runs everywhere in hundreds of different usage profiles.

And really, “desktop Linux” or even “Linux for end-users,” is something of a distraction. We don’t all have to use Linux on the machines beneath our fingers for Linux to be successful. I’m a desktop Linux user because it’s the right system for the work I do, and I can’t work the way I need to with any other kind of system. But I use my systems in a very peculiar way and the thing that makes Linux ideal for me (and the people who are good at building Linux systems,) is not necessarily the qualities that make the best Linux distributions for most users.

As someone who cares about Linux adoption and the use of free software, I don’t want my argument to lead to the very common “let non-technical users use Macs” argument. Although it’s true that OS X can be a convincing introduction to power and use of having a full UNIX-like system on your lap: this was my root (as it were.) Rather, I think that the way to encourage Linux adoption is to increase computer literacy until users respect and value and power that Linux-based systems offer.

Easier said than done, of course.

If this is the case, then Android isn’t a very good introduction to Linux-based operating systems. Not because it’s bad software, but because the kernel is pretty irrelevant to the overall user experience, or the interface that most users have.

Regardless, while madalu is probably right, I don’t think it matters. Android is largely orthogonal to the adoption of Linux. The bigger questions are:

• Does Microsoft have a tablet strategy? Really? The last time Linux made headway into users' hands (i.e. netbooks,) Microsoft changed strategies, and not only pushed Linux-based systems out of the market, but they also basically killed the device class. Netbooks really aren’t a thing anymore.
• How close are we to tablet-like (or tablet-derived) devices from replacing general purpose computers for some classes of day to day activity. I suspect corporate machines will be the first to fall (more constrained/specific use cases; tablet systems give IT administrators more control, and increasingly work happens in web apps.)

If corporate fleets are the first to fall, the first question becomes much more important. In any case, stay tuned, I’m working on collecting the rest of my thoughts on these questions. In the mean time, I’d look forward to hearing from you.

Onward and Upward!

Although I forget it, sometimes, the following video is probably the single best piece of advice for better blogging. Watch the video:

Cory Docotorow - How to be an Uber Blogger

Attention and time are scarce while content is plentiful. “If you write it they’ll read” cannot, thus, be true of blogging. Interesting and important content is interesting, but success lies in managing the attention economy and focusing on output that is easy to read and easy to skim. Its not glamorous, and it requires giving up a bunch of pride, but writers must write with readers in mind.

I’ll skip the meandering analysis and get to a couple of key questions that I think remain open, even 4 years after this video was posted:

• What about non-newsreel blogs? Blogs that are more analysis and less regurgitation’s of boingboing/metafilter/slashdot?
• What about non-blog content? Are books and articles, subject to the same overload (yes?) but are the solutions always “write easy to process, easy to skip,” content? (Maybe?)
• The connection between the “attention” economy, evolving search engine use patterns, and mobile technology change the way that we interact with and compensate for overload?

Discuss!

I wonder if, at some point, this constant state of overload and flux in my world will begin to seem normal and I’ll just adjust to that normal. In the mean time, exciting things are happening and I’m not quite sure of the best way to write about them. Perhaps soon. For now, I’m trying to get better about updating more regularly and I have a bunch of links of stuff that have happened on the wiki in the past couple of weeks that I’d like to share. Here we go:

Discussion of Rhizomes

jfm and I had a good exchange about an old post, /posts/ideology-and-systems-administration. Basically the posts says, “systems administrators have a unique approach to solving technological problems,” and discussed the implications of systems administrators background on technology development. I think our clarifications were useful.

There are a couple of comments on my recent series on a productivity. First, I wrote a post about task planning and creating task items, and Matt posted a comment. Second, a number of us had an ongoing conversation on mobile productivity in response to the “Mobile Productivity Challenges” post that touched on emacs (of course!) input, and context switching.

Site Tweaks

This is a pretty minor point, but I’ve been subtly tweaking the design a little in the site. There are now links to the tags page and the site map in the upper right hand corner. I’ve also made links to as-of-yet-uncreated wiki pages red (according to wiki-convention.) I think (and hope) that red links are easier to spot when they’re red. Feedback on the design would be most welcome. My goal is to make the site welcoming, easy to use, and to minimize the amount of “fussiness.” It might be time for a full refresh, but feedback on the subject might be good.

Critical Futures and Wiki Fiction

Eventually the story will move to the Critical Futures domain, but that’s a bit down the road. Right now I’d rather focus my time/energy on writing some stories, for now (on this wiki.) Infrastructure can come next.

I hope to work on a series of posts that explore collaborative fiction organizing over the next few weeks. If people are interested, that is.

External Links

I came across a blog from the comment on the make emacs better post that I wanted to offer as a link The Babbage Files is a great cyborg/emacs/free software blog.

This is probably not news to anyone, but from John Wiegley I learned about the following two emacs gems that merit mention:

• paredit which makes handling all of the parentheses in Lisp coding much easier.
• redshank, which basically adds a number of tempting systems and associated tools for

And in another direction, I’ve been playing with pylookup. This emacs add on makes it possible to access python documentation from within emacs, from a local copy. The interface is a little bit fiddly but it’s pretty much heaven. More things should work like this.

Onward and Upward!

I’m starting preliminary development on a wiki fiction project that will eventually take over the criticalfutures.com domain. This post is a discussion of that history, my idea, and what I hope to accomplish.

My friend Julia and I have been corresponding on topics related to the future of publishing and genre fiction for a few weeks. When the topic turned to wikis, I spouted off the things I usually say about bootstrapping wiki participation (it’s hard and pretty lonely,) then I had an idea.

I read wikis, mostly wikipeida, a lot. For fun. I’m sure a lot of people do this, as “getting lost in wikipedia,” is a thing that happens. You say, “I’m interested in public transit in Iran,” and you get lost clicking through various pages related to rapid transit systems in the middle east, and then an hour or two is past and you really ought to finish that blog post. The same thing happens on the c2 wiki for me.

While I wish I were less compulsive about it, reading wikis is a pleasurable reading experience, and since the format seems the web, why not run with it? The question becomes: why are we spending so much time figuring out the most ideal way to publish novels and short stories--forms that developed with the physicality of the book--in the digital age?

To be fair, I think there’s a place for digital distribution of paper-centric forms (periodicals and monographs,) but I doubt that in 50 years “digital fiction,” will mean eBook editions of novels. People have been making a similar point for some time about video games for a while. Interactive fiction is definitely a part of digital fiction, but I don’t think it’s the full story.

Meanwhile back at the point…

Here’s the idea. We use wiki software to construct a website that is written as a light-hearted encyclopedia. In the vein of TV Tropes meets Wikipedia except with fictional content. But there needs to be more than just page after page of exposition and condensed blather: my current plan is to have a “dialogue” section, which will be bits of dialogue and scenes published with some contextual metadata (when it happened, who was present, where it happened.) The dialogues can then be linked to as quasi-citations in the more conventional expository wiki pages.

So basically I’m proposing a couple of things here. First, I want to splitting up all content into small self contained pages. This makes it better for multiple people to edit, because editing and writing can happen in a more parallel manner, and you don’t need to agree to an outline, or write things in any sort of sequence. Second, shorter pages with more segmented content is easier to read for the attention limited.

Having said that, I’m not sure that collaborative, for all that will editing is really the way to go. The truth is that so few people edit wikis relative to the number of people who could edit wikis, that you might be better off having some sort of more select editorial community, just in terms of establishing buy-in from contributors and avoiding diffusion of responsibility. I’m undecided.

Along a similar line of thought, I’m considering releasing release updates and new content on a regular basis (e.g. bi-weekly or monthly?) rather than every time an edit is made. This will require some sort of closed-development process. At the same time new wiki projects often fail because there’s little incentive to return to a wiki to “see what’s changed. Blogs, contrast are good at securing return visits.

Thoughts? Anyone interested in being on the editorial board?