Topic: Posts

I don’t know about this. But here it goes, anyway.

I’m pleased to announce the complete publication of my novella “`Knowing Mars <http://tychogaren.com/mars/>`_” on `tychogaren.com <http://tychogaren.com/>`_.

“Knowing Mars” is an important story for me. I wrote it after I graduated from college, after I didn’t go to graduate school the first time, and in a lot of ways it was the project that got me started down the path of being a “real writer,” post-graduation. I’d written fiction before college, and mostly avoided writing fiction in college, and then right as I was finishing college I started writing stories again. It was strange for a while, but it was delightful to be able to tell stories and be so much better at it than I was the first time.

I find that this is fundamentally a recurring issue. When I started writing “Knowing Mars,” I felt like I was starting out light years ahead of what I had written four or five years before. Now, I feel like the stuff I’m working on now is light years ahead of “Knowing Mars.” This is probably an encouraging sign.

I suppose you’d like to know a bit more about the story. I’ll leave most of the details to the reading, but basically it’s a sort of superhero/cyberpunk story that explores themes related to diaspora, political organization, historical narrative, and gender. I don’t know if I’ve ever described it as such before.

The story is available in multiple formats. Each chapter is available in full HTML as part of tychogaren.com and in a plan Markdown formatted plain text. Furthermore the complete text of the novella is available in both of these formats, and a simple un-styled HTML version that should be ideal for conversion to various electronic reading platforms. If you want to read the novella but would find another format easier to process, talk to me about it and I’ll get something pulled together for you.

All “full html” versions of the text have comments enabled using the same system as the blog. I look forward to your comments. Thanks for reading, and stay tuned for more fiction.

Onward and Upward!

This post ties together a train of thought that I started in “The Worst Technologies Always Win” and “Who Wants to be a PHP Developer” with the ideas in the “Ease and the Stack” post. Basically, I’ve been thinking about why the unpopular technologies, or even unpopular modes of using technologies are so appealing and seem to (disproportionately) capture my attention and imagination.

I guess it would first be useful to outline a number of core values that seems to guide my taste in technologies:

• Understandable

Though I’m not really a programmer, so in a lot of ways it’s not feasible to expect that I’d be able to expand or enhance the tools I use. At the same time, I feel like even for complex tasks, I prefer using tools that I can have a chance of understanding how they work. I’m not sure if this creates value in the practical sense, however, I tend to think that I’m able to make better use of technologies that I understand the fundamental underpinnings of how they work.

• Openness and Standard

I think open and standardized technologies are more useful, in a way that flows from “understandable,” I find open source and standardized technology to be more useful. Not in the sense that open source technology is inherently more useful because source code is available (though sometimes that’s true), but more in the sense that software developed in the open tends to have a lot of the features and values that I find important. And of course, knowing that my data and work is stored in a format that isn’t locked into a specific vendor, allows me to relax a bit about the technology.

• Simple

Simpler technologies are easier to understand and easier--for someone with my skill set--to customize and adopt. This is a good thing. Fundamentally most of what I do with a computer is pretty simple, so there’s not a lot of reason to use overly complicated tools.

• Task Oriented

I’m a writer. I spend a lot of time on the computer, but nearly everything I do with the computer is related to writing. Taking notes, organizing tasks, reading articles, manipulating texts for publication, communicating with people about various things that I’m working on. The software I use supports this, and the most useful software in my experience focuses on helping me accomplish these tasks. This is opposed to programs that are feature or function oriented. I don’t need software that could do a bunch of things that I might need to do, I need tools that do exactly what I need. If they do other additional things, that’s nearly irrelevant.

The problem with this, is that although they seem like fine ideals and values for software development, they are, fundamentally unprofitable. Who makes money selling simple, easy to understand, software with limited niche-targeted feature sets? No one. The problem is that this kind of software and technology makes a lot of sense, and so we keep seeing technologies that have these values that seem like they could beat the odd and become dominant, and then they don’t. Either they drop task orientation for a wider feature set, or something with more money behind it comes along, or the engineers get board and build something that’s more complex, and the unpopular technologies shrivel up.

What to do about it?

• Learn more about the technologies you use. Even, and epically if you’re not a programmer.
• Develop simple tools and share them with your friends.
• Work toward task oriented computing, and away from feature orientation.

This post is the culmination of two things:

1. Who wants to be a PHP Developer?

2. An ongoing conversation I’ve had with a number of coworkers about the substandard technologies that always seem to triumph over the “better” options.

The examples of the success of inferior technologies are bountiful. MySQL’s prevalence despite some non-trivial technical flaws (around clustering, around licensing as highlighted by the Oracle merger); PHP as the de facto glue language of the web despite the fact that every other language in it’s class is probably a better programming language (e.g. Python, Perl); VHS and Beta Max; BlueRay (which are proprietary and a physically less durable media) and HD-DVD; and so forth.

The factors are (of course) multiple:

• Marketing.

People have to know about technologies at some stage in their development if the technology is to take off. I’m not sure what that crucial point is, and frankly marketing is something that not only I don’t understand, but I don’t really think anyone understands. Having said that, I think it’s clear that technologies don’t compete simply on their technical merits, and this is in recognition of that.

• Timing.

Technologies that appear at the right time, with regards to availability of alternatives and the needs/interests in the market on those technologies matter a great deal, and can sometimes tip the balance between competing technologies. Arguably MySQL beat PostgreSQL not because it was better, but because it existed in a firm way a little bit earlier. Linux “won” market share over BSD, because BSD wasn’t quite fully free/open source (or available) in 1990-1992 when Linux was taking off.

• Momentum.

A project that doesn’t look like it has energy and a large team behind it is probably doomed to fail on some level, not because it’s a bad technology, but potential users of a technology need to feel confident that it’s going to stick around. If no one is excited about it, then it’ll never win, even if it’s superior in the final analysis.

There’s nothing, really, to be done. I think the truly superior technologies might benefit by paying attention to these factors when it matters, but then the developers of said technologies are probably less interested in marketing and proper timing than the competition. Which is perhaps as it should be.

I guess the lingering questions that I’ll leave you with relate to thinking about the ways that open source and free software relate to other technologies. It strikes me that while there’s a pretty good balance between open source and proprietary technologies in the examples I provided above, all of the open source technologies were commercialized very early on and very intensely in a way that none of their competitors really did. Are “wining technologies,” a mystification of the proprietary technology world? Can community-based open source and free software technologies innovate and “win” in relation to their competitors?

I look forward to sorting out the answers in the comments. Onward and Upward!

So PHP is this programming language that’s widely used, and often reviled by systems administrators and people who fancy theme selves “real programmers.” And yet, I think, while the “real programmers” were busy being “real,” PHP got something very fundamental right that explains its success despite the disdain.

I should interject with some context. First, I think this is another in my ongoing series of posts regarding linguistic relativism and computer programming. Second, for those of you who don’t spend your days in this space PHP is a programming language designed specifically for use in the context of the web, and it has only comparatively recently emerged as a possibility for “general programming tasks,” in contrast to other languages in “the space” (ruby, python, perl, etc.) which started as general purpose languages that have become common for use for web programming. Also as a computer language, there’s nothing particularly innovative about PHP, which earns it no small amount of ire.

So here’s the thing. PHP is easy. It’s designed to be easy. The syntax is familiar to people who are know even a little Perl or other C-like languages. Although the language has had object oriented support for several years, most PHP applications aren’t written in an object oriented manner and in a number of contexts that makes things a bit easier to understand.

And here’s the thing that I seem to notice in the context of administration: compared to other languages and frameworks, PHP is dead simple to deploy. Sure, everything under big loads becomes complex, and sure PHP applications consume more server resources than perhaps they should, but basically you configure a web-server to process PHP code, and then you write your code, inside of your page, and the web server generates what you need it to, and it just works. You don’t have to screw around with writing boilerplate CGI stuff, you don’t have to screw around with cgi-bins/ and script aliases which were never intuitive, you don’t need special servers, it just works.

And I already know that, someone is going to tell me that there’s a Perl module that lets you use perl in the same way, or that Python and Ruby don’t make you write CGI boilerplate either (or that there’s a Perl module to write the CGI boilerplate). And I know these things, but I’m not sure that it matters anymore. PHP, as a language is written around the needs of web development, and there’s merit in that.

I’m not saying, “go forth and write your next application in PHP:” I don’t even know if dynamic web applications are worth writing anymore. I am saying that despite all of the dreck in the PHP space, there are some things that are incredibly worthwhile that the current generation of web developers may miss.

That is all. For now. Onward and Upward!

As if I needed a new project, this post introduces a new project that’s floating around in my mind. I was having a conversation with a friend about how I use the computer, I realized that while I’ve talked about various elements of how I use computer’s (the short story: peculiarly), I’ve not really talked about the holistic experience. As I started to talk about the various components and how they connect and work together, I realized that with out an example it was about as clear as mud.

So, in light of this, I’ve decided to make a “tychoish stack,” which won’t be anything particularly novel, but a repackaging of the software--mostly configurations and little bits here and there--that I use on a daily basis. My stumpwm configuration. The highlights of my Emacs configuration, and a few install scripts to make it all work together. An SSH configuration file that will make your life much easier, a list of packages that you’ll want to install on common operating systems (Debian/Ubuntu and Arch Linux), and--because I am who I am--a fair piece of writing about best practices and how to these tools effectively.

I was talking with Chris the other day in one of our never ending conversations regarding is ever changing choices of desktop operating systems. “Windows just feels more polished to me right now,” he said after a stint with the latest Ubuntu or one of its derivatives.

To which I said, “of course it does,” they pay bunches of people lots of money to make sure that Windows is polished and it’s a high priority given the failure of Vista and the direction of the market. The reason I use Linux full time is not because I want a better more polished experience, I use Linux full time, because I want something very specific: a window manager that stays out of my way and doesn’t distract me with “chrome,” emacs buffers that run in the way that I expect them to, package management tools that allow my system to work and function day in and day out, the ability to customize all of these functions to suit the evolving needs of my work, and nothing else that I have to mess around with.

This is something that I can only get from a UNIX system, and the more I play with different systems, the more I’m inclined to think that the only way to get this is with Arch Linux. But that’s just me. I’ve played with a bunch of different operating systems (that’s an aspect of my day job) and I’ve spent time using OS X, and it just doesn’t work for me. I don’t need smooth, I don’t want polish, I just want something that lets me work.

Different computer systems make sense for different people. There’s no problem with that assertion, I think.

The problem of course, that my setup doesn’t scale. I can go from a bare arch installation to a working version of my system in a few hours by using rsync to copy over my home directory, updating a few git repositories, installing a list of packages, and creating a half dozen symbolic links, but building this from the bottom up would take a long time.

The goal of this project then, is to make that process easier. I’ve done a bunch of work to get a setup that does what I need it to do, I know which applications work, I know how to plug everything together to make it easier to manage. I want to build a stack so that you all can take it, learn from what I’ve done, and spend the time customizing it to what you do, rather than going through the trouble of building it up yourselves.

How’s that sound?

In many ways, I think you could say, I live and work in a bubble of the technical future that, as Gibson said “isn’t evenly distributed,” yet. I have developed a set of tools and work flows that enable me to work nearly anywhere and on a moment’s notice. I work for a company which great and open internal infrastructure that allows us to securely communicate and collaborate in whatever way we think will best serve the projects we’re working on. And I know enough to be able to automate the boring parts of my technological experience. In all, pretty good.

Here’s the thing though: despite all of this technological infrastructure, all this know how and frankly awesome connectivity: all of the tools we use to collaborate technologically: chat rooms, wikis, paste-bins, version control systems, instant messages and email all work better when you’re in the same room. Examples:

I was sitting in a talk with a coworker, and we were both logged into an IRC room from our laptops, where we were able to share some useful examples, links, and other commentary without being (very) disruptive. In day to day work, I (and my coworkers) spend a lot of time using chat rooms to communicate and share information with people who are only a few feet away, and in the end we get a lot done.

There are probably a lot of reasons why this is the case: digital relationships are almost always supported by real life relationships, there’s a level of hard to document interstitial and context setting that we do in real life that is difficult to efficiently create digitally, but that can be accomplished without second thought *in real life, and so forth. But, having made this realization, I think there are a few conclusions to be drawn about collaboration technology:

• It helps to centralize information flow. So much collaboration technology is “pull” based, and there’s no good way to ensure that people know you’ve done something that they might consider without pushing information to them in some manner. Even so, create one place where the people you’re working with can see what you’re working on.

  Use something like an IRC channel, or an xmpp MUC room, combined with a service like notifixious or something similar. In a lot of ways, the incessant emails Facebook sends achieve the same goal.

• Communities come together to work on something specific and concrete, but inevitably they bond and endure for other reasons and other kinds of conversations. While creating “off topic” silos is awkward, creating the space for people to get to know each other is essential to making people work together well (and thus use collaborative technology better.)

• Focus most of your attention on “getting things done,” and less attention on the “how things are done.” There are so many technological solutions, so many options, and so many different contexts that it doesn’t really matter how things get done as long as they do get done. The right and preferred tools will arise and present themselves when needed, and as long as things are getting done using the right tool doesn’t matter much.

Onward and Upward!

I’ve been trying to figure out wikis for a long time. It always strikes me that the wiki is probably the first truly unique (and successful) textual form of the Internet age. And there’s a lot to figure out. The technological innovation of the wiki is actually remarkably straightforward,¹ and while difficult the community building aspects of wikis are straightforward.² The piece of the wiki puzzle that I can’t nail down in a pithy sentence or two is how to organize information effectively on a wiki.

That’s not entirely true.

The issue, is I think that there are a number of different ways to organize content for a wiki, and no one organizational strategy seems to be absolutely perfect, and I’ve never been able to settle on a way of organizing wiki pages that I am truly happy with. The goals of a good wiki “information architecture” (if I may be so bold) are as follows:

• Clarity: It should be immediately clear to the readers and writers of a wiki where a page should be located in the wiki. If there’s hierarchy, it needs to fit your subject area perfectly and require minimal effort to grok. Because you want people to focus on the content rather than the organization, and we don’t tend to focus on organizational systems when they’re clear.
• Simplicity: Wikis have a great number of internal links and can (and are) indexed manually as needed, so as the proprietor of a wiki you probably need to do a lot less “infrastructural work” than you think you need to. Less is probably more in this situation.
• Intuitive: Flowing from the above, wikis ought to strive to be intuitive in their organization. Pages should answer questions that people have, and then provide additional information out from there. One shouldn’t have to dig in a wiki for pages, if there are categories or some sort of hierarchy there pages there shouldn’t be overlap at the tips of various trees.

Strategies that flow from this are:

• In general, write content on a very small number of pages, and expand outward as you have content for those pages (by chopping up existing pages as it makes sense and using this content to spur the creation of new pages.
• Use one style of links/hierarchy (wikish and ciwiki fail at this.) You don’t want people to think: Should this be a camel case link? Should this be a regular one word link? Should this be a multiple word link with dash separated words or underscore separated words? One convention to rule them all.
• Realize that separate hierarchies of content within a single wiki effectively create separate wikis and sites within a single wiki, and that depending on your software, it can be non-intuitive to link between different hierarchies.
• As a result: use as little hierarchy and structure as possible. hierarchy creates possibilities where things can go wrong and where confusion can happen. At some point you’ll probably need infrastructure to help make the navigation among pages more intuitive, but that point is always later than you think it’s going to be.
• Avoid reflexivity. This is probably generalizable to the entire Internet, but in general people aren’t very interested in how things work and the way you’re thinking about your content organization. They’re visiting your wiki to learn something or share some information, not to think through the meta crap with you. Focus on that.
• Have content on all pages, and have relatively few pages which only serve to point visitors at other pages. Your main index page is probably well suited as a traffic intersection without additional content, but in most cases you probably only need a very small number of these pass through pages. In general, make it so your wikis have content everywhere.

… and other helpful suggestions which I have yet to figure out. Any suggestions from wiki maintainers?

My most sincere apologies to Virginia Woolf for the title.

We use a lot of git at work, and I’ve earned a bit of a reputation as “the git guy,” both at work and amongst the folks who read the blog. So, I suppose it should come as no surprise that a coworker (hi stan!) said “You should write something about using git when it’s just one person.” And I said “well sure, but it’s not nearly as interesting as you think it’s going to be.” He didn’t seem to mind, so here I am.

Lets back up for a second.

Git is a tool that programmers use to facilitate collaboration. It stores versions of computer code (and associated file) and save incremental sets of changes to those files, so that programmers can easily experiment with changes without destroying code, and so that teams of programmers (sometimes even large teams) can all work and develop on a single code base without stepping on eachothers toes, or duplicating efforts because you end up working on different versions of the code.

Git makes a number of innovations that make version control with git much preferable (at least in my experience) to other tools, but fundamentally that’s what git does. Git has all sorts of innovations that make it awesome: it’s fast, it can take “diverged branches” and merge them together painlessly and almost automatically. It’s great and mind bending, and I think really forces us to rethink all sorts of assumptions about authorship, and the coherency of “texts” in general.

But I’m famous for using git all alone, with just me. Here are the lessons and conclusions that I’d draw from my experiences over the past… two or three (or so) years of using git:

• Use fewer features. Git can do all sorts of funny stuff with branches and merges, but the truth is that when you’re working alone you don’t really want to have to much with branches. Because they’re the really novel feature (at least in terms of their usability) in git, everyone wants to use them but they add complexity, and there are other approaches to managing files and content with git that are probably preferable.
• Resist the temptation to store binary files in git. It’ll work, but you won’t be really happy with it.
• Even though you don’t need to have a remote repository to push your git repositories to, keep an off site repository in almost every case. You get incremental backups for free with git, and remote back ups are nearly free.
• Use a tool like gitosis (but it’s probably in your distribution’s repository and you should use that version) to manage repositories. It’s overkill for your use-case, but it makes things easier in terms of creating repositories. Perhaps consider something like girocco if you want even more overkill, and more web-based interface.
• There are git tools for most text editors and graphical tools that you may choose to use, but don’t, at least until you understand what’s going on behind the scenes. Learn git commands, and do stuff from the command line, as you’ll be much better (in the long term,) at fixing things as issues come up.
• If you need to maintain multiple machines, think of each machine as a collaborator, and it’s probably easiest to have a centralized group of repositories that you can push to in order to keep these machines up to date.
• If you’re using git to manage configuration files (which is great) I strongly recommend having a “sub-home” directory in a git repository with your configuration files with symbolic links pointing to the files in the repository. This strikes me as towing the balance between utility and control, without being a total pain the ass. As it were.

And that’s about it.

The background:

Sun Microsystems developed this file system called “ZFS,” which is exceptionally awesome in it’s capabilities and possibilities. The problem, is that it was developed and released as part of the Open Solaris project which has a licensing incompatibility with the Linux Kernel. Both are open source, but there is a technical (and not all together uncommon) conflict in the terms of the license that makes it possible to combine code from both licenses in a single executable.

Basically the GPL, under which the Linux Kernel is distributed, says if you distribute a binary (executable) under the terms of the GPL, the source code is all files that you used to make that binary. By contrast ZFS’s license says “here are all the files that we used to make this binary, if you change them when you make your binary and give that binary to other people you have to give them, but if you add additional files, you don’t have to give those out to people.”

Apparently the idea behind the ZFS license (i.e. the CDDL, and the MPL from whence it originated,) is that it allows for easier embedding of ZFS (and other technologies) in proprietary code because the resulting binary isn’t list covered by the CDDL in most cases. Even though the CDDL is incredibly confusing, apparently it’s more “business friendly,” but I diverge from my original point.

And so if Linux users want to run ZFS, they have to run it as a user-space process (i.e. not in the kernel,) which is suboptimal, or they have to run Solaris in a vitalized environment (difficult,) or something. There’s also a ZFS-like file system called “btrfs,” which can be included in the kernel (interestingly, developed by Oracle who of course now own ZFS itself,) but it is not production ready.

What I’m about to propose is an end run around the GPL. Because it seems to me that combining the source code violates neither license, distributing source code violates no license. Compiling the source code for your own use violates no license. I mean it’s annoying and would require a bit of bootstrapping to get a Linux+zfs system up and running, but this is the kind of thing that Gentoo Linux users do all the time, and isn’t a huge technological barrier.

It feels a bit creepy of course. but I think it works. The logic has also been used before. We’ll call it the “PGP loophole.”

PGP is an encryption system that’s damn good so good in fact, that when it was first released, there were export restrictions on the source code because it qualified as military-grade munitions in America. Source code. How peculiar. In any case there were lawsuits, and PGP source was released outside of America by printing it in a book. Which could be disassembled and scanned into a computer and then compiled. Books were never and--as far as I know--are not classified as munitions, and so they could be exported. Of course I’m not a lawyer, but it strikes me that linux+zfs and PGP in the 90’s may be in analogous situations.

And I think, because this proposal centers around the distribution of source code and only source code this kind of distribution is fully within the spirit of free software. Sure it’s pretty easy, even for the “good guys,” to run a foul by distributing a binary, but this would be easy to spot, and there are already suitable enforcement mechanisms in place, for the Linux kernel generally, and Oracle’s legal department which we can assume will take care of itself.

Or Oracle could release ZFS under GPL. Either solution works for me.

A reader (hi grandma!) asked me to write a post about why I’m so interested in open source, and who am I to refuse. In fact, I tend to do requests pretty well, so if there’s a subject you’d like to see me cover here, just ask and I’ll see what I can do. In any case, I’ve been involved, for varying definitions of involved, in Free Software and open source for a few years now. On a personal level, I use this software (almost exclusively) because I can make it do exactly what I need it to do, because it’s very stable, and because from an architecture perspective I understand how these systems work and that’s useful for me. Having said that, I think open source is important and worth considering for reasons beyond the fact that I (and people like me) find it to be the most important tool for the work we do.

When folks get together and say “I’m going to work on an open source project,” I think some interesting things happen. First, they’re making a number of interesting economic decisions about their work. There are business models around open source, but they are more complex than “I make software, you give me money for software,” and thus require people to think a little bit more widely about the economic impact of their work. I think the way that people view the implications of their labor is incredibly important, and free software presents an interesting context to think about these questions.

The second, and perhaps larger reason I’m interested in open source is the community. Open source developers often know that the things they want to create are beyond the scope of their free time and personal ability, so they collaborate with other people to make something of value and worth. How this collaboration happens: what motivates developers, how they create tools and technologies to support this kind of work flow, how the “intellectual property” is negotiated (particularly in projects that don’t use the GNU GPL,) how leaders are selected and appointed, how decisions are made as a community, and how teams are organized and organize themselves. These are intensely fascinating.

And these phenomena matter, both in and for themselves, but also as they impact and connect with other questions and phenomena in the world. For instance:

• I think that the decision making process in free software projects is instructive for thinking about how all sorts of communities can reach a “decision making fork” resolve it somehow and then continue with their work. Some open source projects have formal structures, and that is easier to understand from the outside, but most make decisions in an informal way, and that decision making process is potentially novel, I’d argue. In what other context do people have to construct projects outside of work.
• While leaders in the open source community are rarely elected (aside from a number of notable examples; the Debian Project Leader springs instantly to mind) most projects are very democratic. But this requires that we keep in mind a fairly broad definition of democracy. Because there isn’t a lot of voting, and sometimes decisions aren’t discussed thoroughly before people start doing things, it doesn’t look democratic. But everything is volunteer based, and leaders I think have a sense of responsibilities to their constituencies, which is meaningful.
• The tools that open source developers use are, unsurprisingly open source, and are often picked up and used by teams that aren’t making free software. I’m interested in thinking about how “ways of working,” proliferate out of open source and into other spheres. Is non-open source developed differently if the developers are familiar with and use open source tools?
• Similarly, I think I’m interested in thinking about how the architecture of Linux and Unix give rise to a thought about APIs and open standards in a way that doesn’t necessarily happen on closed platforms. After a certain point, I think I’m forced to ask: is GNU/Linux the leading free software/open source platform because it just happens to be, or because it’s UNIX. Is there something special about the design of UNIX that leads to openness, and the practices of openness? To what extent does the limitations of the environment (operating system here) the social conventions that are built on it?

And then beyond the specific questions--which are terribly important in and of themselves--open source present a terribly exciting subject for the study of these issues. There is so much data on the ground concerning open source: version control history, email logs, IRC logs, and so forth. Not only are the issues important but the data is rich, and I think has a lot to tell us if we (I?) can bother to spend some time with it.