dialectical futurism

Gear Mongering

2010-07-27T00:00:00-04:00

At the end of the day (or even the beginning,) I’m just another geek, and despite all of the incredibly (I’d like to think at least) reasoned ways I think about the way I use technology, I occasionally get an old fashioned hankering for something new. We’ve all had them. Perhaps my saving graces are that I do occasionally need new things (computers wear out, cellphones are replaced, needs change), and the fact that I’m both incredibly frugal and task oriented (rational?) about the way I use technology.

But I’m still a geek. And gear is cool. Thoughts, in three parts.

Part One, Phones and the HTC EVO

I’ve been using a Blackberry for 18 months, and I’ve come to two conclusions: blackberries are great and they have the “how ot integrate messaging into a single usable interface.” I was skeptical at first, and it’s very simple, and I’ve never quite gotten my email to function in an ideal way, largely because I think the Blackberry works really well for email. It’s everything else that I might want to do with my phone that I can’t, and I’d probably like to: I have an SSH client and it’s nearly usable. Nearly. I have IM clients, that are nearly functional. Nearly.

When I got the Blackberry, is and was the most important communication I was doing. I worked for a very email-centric company, and I wanted to be able to stay in the email-loop even when I was off doing something else. These days, IRC and XMPP are a far more central feature of my digital existence, and I tend to think that it’s not an Internet connection if I can’t open SSH. I’m also switching on a much longer public-transit focused commute in the next few week, and being able to do research for writing projects will be nice. I’m not sure what the best solution is exactly, though the HTC EVO is a pretty swell phone.

As the kids these days say, “Do want.”

Part two, Infrastructural Computing and Home Servers

I’ve fully adopted an infrastructural approach to technology, at least with regards to my own personal computing. That was a mouthful. Basically, although I work locally on the machine that’s in front of me (writing, email, note taking, collaboration,) much of the “computing” that I interact with isn’t actually connected to the machine I interact with directly. In some ways, I suppose this is what they meant when they said “cloud computing,” but the truth is that my implementation is somewhat more… archaic: I use a lot of SSH, cron, and a little bailing wire to distribute computing tasks far and wide, and the process of moving everything in my digital world from a laptop that I carried around with me everywhere (college,) to a more sane state of affairs has been a long time coming.

Right.

The long story short is that aside from a machine (my old laptop) that’s at capacity powering my “stereo,” I don’t have any computer’s at home aside from my laptop, and I tend to take it everywhere with me, which makes it unideal for some sorts of tasks. Furthermore, without an extra machine setting around, file storage, some kinds of backups, are somewhat more complicated than I’d like. So, I’m thinking about getting some sort of robust server-type machine to stick in a corner in my apartment.

Not exactly sure what the best option is there. I’m burdened by: frugality, sophisticated tastes, and the notion that having quality hardware really does matter.

More thinking required.

Part three, More Laptops

So I might have a laptop related illness. Back in the day, laptops always seemed like a frivolity: underpowered, never as portable as you wanted, awkward to use, and incredibly expensive. Now, laptops are cheap, and even the Atom-based “netbooks,” are functional for nearly every task. I tend to buy used Thinkpad Laptops, and as I think about it, I’ve probably spent as much on the three Thinkpads, all of which are still in service, as I did on any one mac laptop.

The thing about my current laptop is that when you think about it, it’d make a decent home server: the processor has virtualization extensions, the drive is fast (7200 rpm) and it can handle 4 gigs of ram (and maybe more.) What more could I want? And I distributed things correctly, the “server” laptop could be pressed into service as a backup/redundant laptop, in case something unforeseen happened.

Or I could dither about it for another few months, and come to some other, better, fourth solution.

Onward and Upward!

Creating Useful Archives

2010-07-22T00:00:00-04:00

I’ve done a little tweaking to the archives for dialectical futurism recently, including creating a new archive for science fiction and writing and being who I am this has inspired a little of thought regarding the state and use of archives of blogs.

The latest iteration of this blog has avoided the now common practice of having large endless lists of posts organized by publication month or by haphazardly assigned category and tag systems. While these succeed at providing a complete archive of every post written, they don’t add any real value to a blog or website. I’m convinced that one feature of successful blogs moving forward will be archives that are curated and convey additional value beyond the content of the site.

Perhaps blogs as containers for a number of posts will end up being to ephemeral than I’m inclined to think about them, and will therefore not require very much in the way of archives, Perhaps, Google’s index will be sufficient for most people’s uses. Maybe. I remain unconvinced.

Heretofore, I have made archives for tychoish as quasi-boutique pieces: collections of the best posts that address a given topic. This is great from the perspective of thinking about blog posts as a collection of essays, but I’ve started to think that this may be less less useful if we think of blogs as a collection of resources that people might want to have access to beyond it’s initial ephemeral form.

Right now my archives say “see stuff from the past few months, and several choice topics on which I wrote vaguely connected sequences of posts.” The problem with the list of posts from the last few months is that beyond date, there’s not a lot of useful information beyond the title and the date. The problem with the topical archives is that they’re not up to date, their not comprehensive even for recent posts, and there’s little “preview” of a given post beyond it’s title. In the end I think the possibility of visiting a topical archive looking for a specific post and not finding it is pretty large.

In addition to editorial collecting, I think archives, guides, or indexes of a given body of information ought to, provide some sort of comprehensive method for accessing information. There has to be some middle ground.

I think the solution involves a lot of hand mangling of content, templates, and posts. I’m fairly certain that my current publication system is probably not up for the task without a fair amount of mangling and beating. As much as I want to think that this is an problem in search of the right kind of automation, I’m not sure that’s really the case. I’m not opposed to editing things by hand, but it would increase the amount of work in making any given post significantly.

There is, I suspect, no easy solution here.

In Favor of Simple Software

2010-07-13T00:00:00-04:00

I’ve spent a little bit of time addressing some organizational and workflow angst in the past few weeks, and one thing I’d been focusing on had been to update and fine tune my emacs (text editor) and irssi (irc/chat) configuration. Part of my goal had been to use irssi-xmpp to move all of my chat/synchronous communication into one program; unfortunately I’ve not been able to get irssi-xmpp to build and function in a fully stable way. This is probably because I’m hard on software and not because of anything specific to the software itself.

In any case, this lead me to come to the following conclusion about these programs, as they are probably the two most central and most heavily used applications in my arsenal, and without a doubt are the applications that I enjoy using the most. I scribbled the following note a few days ago in preparation for this entry:

In many ways the greatest advance or feature that these programs provide isn’t some killer feature, it’s a simple but more powerful abstraction that allows users to interact with their problem domain. Emacs is basically a text-editing application framework, and provides users with some basic fundamentals for interacting with textual information, and a design that allows users to create text editing modalities or paradigms which bridge the divide between full blown applications and custom configurations. By the same token, Irssi is really a rather simple program that’s easy to script, and contains a number of metaphors that are useful for synchronous communication (chat.)

And we might be able to expand this even further: these are two applications that are not only supremely functional, but are so usable because they are software projects that really only make sense in context of free software.

I want to be very careful here: I don’t want to make the argument that free software isn’t or can’t be commercial, because that’s obviously not the case. At the same time, free software, like these applications needn’t justify itself in terms of “commercial features,” or a particular target market in order to remain viable. It’s not that these programs don’t have features, it is that they have every feature, or the potential for every feature, and are thus hard to comprehend and hard to sell. Even if it only takes a use over a short period of time for users to find them incredibly compelling.

The underlying core extensibility that both of these “programs” have is probably also something that is only likely to happen in the context of open source or free software. This isn’t to suggest that proprietary software doesn’t recognize the power or utility of extensible software, but I don’t think giving users so much control over a given application makes sense from a quality control perspective. Giving users the power to modify their experience of software in an open ended fashion, also gives them the power to break things horribly, and that just doesn’t make sense from a commercial perspective.

There’s probably also some hubris at play: free software applications, primarily these two, are written by hackers, with a target audience of other hackers. Who needs a flexible text editing application framework (e.g. emacs), but other programmers. And the primary users of IRC for the past 8-10 years have largely been hackers and developers and other “geek” types. irssi is very much written for these kinds of users. To a great extent, I think it’s safe to suggest that when hackers write software for themselves, this is what it looks like.

The questions that must linger is: why isn’t other software like this? (Or is it, and I’m missing it in my snobbishness,) and where is the happy medium between writing software for non-hackers and using great software (like these) to “make more hackers.”

Onward and Upward!

Focus and Context Switching

2010-07-08T00:00:00-04:00

This post is inspired by Cory Doctorow’s Inventory about the technology he uses for his work and ongoing personal angst and gadget lust.

For the past six months or so I collapsed my entire computing existence into one, single, computer. It’s a nifty Thinkpad x200, a small laptop with just the right balance of usable screen space, sufficient computing power, and great build quality. And when I say “everything,” I mean it. I hook it up to a monitor and keyboard at the office and do my day-job (technical writing/sys admin stuff) on this system, I have a similar “desktop” situation at home, and I write fiction, do all of my email, write and blog posts, off of this system. I even have development web servers running here.

In a number of ways it’s great, and I wouldn’t trade this for the world. Everything just works the way I want it to, and I never have to worry that I’ve left some important edits to a file on a system elsewhere. Everything is always with me.

Now to be fair, I have additional computers. My old(der) desktop at home keeps backups of files, plays music, and does a number of other tasks. I have (and use) the server that this and other websites run on for some tasks, and I have another instance at the office that manages some work functions, but despite their varying physical distance from me at any given point, my interactions with these computers is always as if they’re remote. When I use a computer, it’s this one.

Now there isn’t a real problem here, except that from a workspace and mindspace perspective the context switching can be somewhat complicated and frustrating. While I’ve got most of the kinks worked out of the docking (monitors and keyboards) process figured out, it takes me a few moments to settle into or out of “laptop-mode” or “workdesk-mode” or “homedesk-mode.” While not having to worry if my files are all up to date, it’s also somewhat distracting for all my different projects to be open all the time. The article I’m working on for work is always open and a few key presses away from the novel I’m working on or the latest in-progress blog post.

Again, this isn’t a really huge issue, but it means that when I get somewhere and want to begin working I have to take a deep breath, and spend a moment or two getting going again. These do fit into the category of “first world problems,” and I’m not sure if there’s a really easy solution. I’ve toyed with a number of resolutions to this angst:

Distribute my existing machines such that I have a machine that lives at work and a machine that lives at home so that I can just sit down at a desk and start working without shenanigans. Given the available hardware, this might mean that I’d spend most of my time using systems that I’m not particularly fond of.
I’ve toyed with having a “tycho writing laptop” that wouldn’t have a web browser installed, for more distraction free writing. I’ve got my old laptop set up to do this, but I’m not likely to take it anywhere (i.e. for the commute,) so it might make sense to get a little netbook for the train for this function, but that seems like overkill.
I might redistribute more of my workload to servers rather than doing everything on the laptop. I’m thinking about having the terminal sessions that I use for email primarily live in screen sessions elsewhere.
I’ve thought about getting a second laptop, (like my main system at the moment,) both for redundancy and to help reduce the cost of switching between various contexts.

As I’ve been toying with this, I’ve been making a number of tweaks to my work flow to help address the pain of context switching, most of which are too trivial and too specific to outline here. Mostly I’ve been tweaking some customizations, improving how I use virtual desktops. While these tweaks have improved things greatly, better internal system management doesn’t solve the underlying issues: it takes time to reconnect to networks, to close tabs in the web browser, to get to the relevant files open in emacs, and navigate to the proper desktop. All the while other contexts (other files, other virtual desktops,) lurk nearby.

And figuring out how to solve this problem involves a certain amount of “head game” for me: avoiding having “the old laptop” be the primary computer for a given task, making sure that I don’t need a network connection for essential tasks. Assorted other weirdnesses.

If anyone recognizes features of this angst that in their own work they’ve managed to resolve, I’d love to hear about your setup.

Fan Fiction is Criticism

2010-07-06T00:00:00-04:00

Thanks to Shaun Duke for inspiring this little rant.

I must confess that I’m mostly uninvolved in the world of fan fiction these days, though I have traveled in “fanish” circles at various points in my past. It’s not because I don’t think fans have interesting things to say about literature and media, or that I don’t think what’s happening in fandom important and fascinating. No, I’m mostly withdrawn because I have too much on my plate and participating in fandom doesn’t really contribute to the specific goals I have at this moment. But I sometimes feel that way about social science.

In any case, I’d like to put forth the following arguments for viewing fan fiction as a form a literary criticism rather than a literary attempt in it’s own right:

Fan fiction is a form of literary criticism. Sure it’s casual, sure it’s written in the forum of a story, but the fan fictioner and the critic both write from the same core interest in interpreting texts and using varying readings of texts to create larger understandings of our world.
The fact that fan fiction looks like a story, is mostly distracting to what’s happening in these texts. Fan fiction, has always been written in communities. The people who read fan fiction are largely the people who write fan fiction. Fan fiction inspires
The quality of fan fiction is also largely irrelevant to the point of whether fan fiction is worthwhile. More so than other forms of writing, fan fiction is less about the technical merits of the text, and more about the discursive process under which the texts are created. Better quality writing makes better fan fiction, certainly but I don’t think fan fiction centers on those kinds of values.
Copyright, and the “intellectual property” status of fan fiction is also sort of moot. It’s true that if we’re being honest fan fiction impinges upon the copyright of the original author. At the same time, fan fiction doesn’t really hurt creators: people aren’t confused that fan fiction is “real fiction,” fan fiction by and large doesn’t divert sales from “real fiction,” and so forth. Sure, it’s a bit weird for some others to find other people playing in their sand boxes, but the truth is that authors have never had a great deal of control over what happens to their work post-publication, so it’s fair.

Additionally, I think that fan fiction accomplishes something that are incredibly powerful and worthwhile that “normal” fiction cannot accomplish. Writing fan fiction can be, I’d wager, an incredibly effective educational experience for new writers, particularly genre fiction writers. By providing a very fast feedback loop with an audience of readers and writers (and lovers of literature and story telling.) Not to mention the fact that because fan fiction tends to be somewhat ephemeral and there’s a wealth of inspiration and impetus for fiction, fan writers can write a lot, and if they choose in a very productive sort of way.

And that is almost certainly a good thing.

The Overhead of Management

2010-07-01T00:00:00-04:00

Every resource, every person, every project, every machine you have to manage comes with an ongoing cost. This is just as true of servers as is it is of people who work on projects that you’re in charge of or have some responsibility for, and while servers and teammates present very different kinds of management challenges, working effectively and managing management costs across contexts is (I would propose) similar. Or at least similar enough to merit some synthetic discussion.

There’s basically only one approach to managing “systems administration costs,” and that’s to avoid it as much as possible. This isn’t to say that sys admins avoid admining, but rather we work very hard to ensure that systems don’t need administration. We write operating systems that administer themselves, we script procedures to automate most tasks as much as possible (the Perl programing language was developed and popularized for use of easing the administration of UNIX systems,) and we use tools manage larger systems more effectively.

People, time, and other resources cannot be so easily automated, and I think in response there are two major approaches (if we can create a somewhat false dichotomy for a moment:)

On the one hand there’s the school of thought that says “admit and assess management costs early, and pay them up front.” This is the corporate model in many ways. Have (layers upon layers of) resources dedicated to managing management costs, and then let this “middle management” make sure that things get done in spite of the management burden. On servers this is spending a lot of time choosing tools, configuring the base system, organizing the file system proactively, and constructing a healthy collection of “best practices.”

By contrast, the other perspective suggests that management costs should only be paid when absolutely necessary. make things, get something working and extant and then if something needs to be managed later, do it then and only as you need. On some level this is inspiring philsophy behind the frequent value of favoring “working code” over “great ideas” in the open source world.¹ Though I think they phrase it differently, this is the basic approach that many hacker-oriented start ups have taken, and it seems to work for them. On the server, this approach is the “get it working,” approach, and these administrators aren’t bothered by having to go in every so often to “redo” how things are configured, and I think on some level this kind of approach to “management overhead” grows out of the agile world and the avoidance of “premature optimizations.”

But like all “somewhat false dichotomies,” there are flaws in the above formulation. Mostly the “late management” camp is able to delay management most effectively by anticipating their future needs (either by smarts or by dumb luck) early and planning around that. And the “early management” camp has to delay some management needs or else you’d be drowned in overhead before you started: and besides, the MBA union isn’t that strong.

We might even cast the “early management” approach as being “top down,” and the “late management” camp as being “bottom up.” If you know, we were into that kind of thing. It’s always, particularly in the contemporary moment to look at the bottom-up approach and say “that’s really innovative and awesome, that’s better,” and view “top-down” organizations as “stodgy and old world,” when neither does a very good job of explaining what’s going on and there isn’t inherent radicalism or stodginess in either organization. But it is interesting. At least mildly.

Thoughts? Onward and Upward!

Alan Cox’s Cathedrals, Bazaars and the Town Council
↩

Jekyll Publishing

2010-06-29T00:00:00-04:00

I wrote about my efforts to automate my publishing workflow a couple of weeks ago, (egad!) and I wanted to follow that up with a somewhat more useful elucidation of how all of the gears work around here.

At first I had this horrible scheme setup that dependent on regular builds triggered by cron, which is a functional, if inelegant solution. There’s a lot of tasks that you can give the appearance of “real time,” responsiveness by scheduling more brute tasks regularly enough. The truth is, however, that its not quite the same, and I knew that there was a better way.

Basically the “right way” to solve this problem is to use the “hooks” provided by the git repositories that I use to store the source of the website. Hooks, in this context refer to a number of scripts which are optionally run before or after various operations in the repositories that allow you to attach actions to the operations you perform on your git repositories. In effect, you can say “when I git push do these other things” or “before I git commit check for these conditions, and if they’re not met, reject the commit” and so forth. The possibilities can be a bit staggering.

In this case what happen is: I commit to the tychoish.com repositories a script that synchronizes the appropriate local packages runs and publishes changes to the server. It then sends me an xmpp message saying that this operation is in progress. This runs as the post-commit hook, and for smaller sites could simply be ”git push origin master”. Because tychoish is a large site, and I don’t want to be rebuilding it constantly, I do the following:

 #!/bin/bash 

 # This script is meant to be run in a cron job to perform a rebuilding
 # of the slim contents of a jekyll site. 
 #
 # This script can be run several times an hour to greatly simplify the
 # publishing routine of a jekyll site.

 cd ~/sites/tychoish.com/

 # Saving and Fetching Remote Updates from tychoish.com
 git pull >/dev/null && 

 # Local Adding and Committing
 git checkout master >/dev/null 2>&1
 git add .
 git commit -a -q -m "$HOSTNAME: changes prior to an  slim rebuild"  >/dev/null 2>

 # Local "full-build" Branch Mangling
 git checkout full-build >/dev/null 2>&1 &&
 git merge master && 

 # Local "slim-bild" Branch Magling and Publishing
 git checkout slim-build >/dev/null 2>&1 &&
 git merge master && 
 git checkout master >/dev/null 2>&1 &
 git push --all

 # echo done

Then on the server, once the copy of the repo on the server is current with the changes published to it (i.e. the post-update hook), the following code is run:

 #!/bin/bash
 #
 # An example hook script to prepare a packed repository for use over
 # dumb transports.
 #
 # To enable this hook, make this file executable by "chmod +x post-update".

 unset GIT_DIR
 unset GIT_WORKING_TREE

 export GIT_DIR
 export GIT_WORKING_TREE

 cd /path/to/build/tychoish.com
 git pull origin; 

 /path/to/scripts/jekyll-rebuild-tychoish-auto-slim & 

 exit

When the post-update hook runs, in runs in the context of the repository that you just pushed to, and unless you do the magic (technical term, it seems) the GIT_DIR and GIT_WORKING_TREE variables are stuck in the environment and the commands you run fail. So basically this is a fancy git pull, in a third repository (the one that the site is built from.) The script jekyll-rebuild-tychoish-auto-slim looks like this:

 #!/bin/bash
 # to be run on the server

 # setting the variables
 SRCDIR=/path/to/build/tychoish.com/ 
 DSTDIR=/path/to/public/tychoish/
 SITENAME=tychoish
 BUILDTYPE=slim
 DEFAULTBUILD=slim

 build-site(){
      cd ${SRCDIR}
      git checkout ${BUILDTYPE}-build >/dev/null 2>&1
      git pull source >/dev/null 2>&1
      
      /var/lib/gems/1.8/bin/jekyll ${SRCDIR} ${DSTDIR} >/dev/null 2>&1
      echo \<jekyll\> completed \*${BUILDTYPE}\* build of ${SITENAME} | xmppipe garen@tychoish.com

      git checkout ${DEFAULTBUILD}-build >/dev/null 2>&1
 }

 build-site;

This sends me an xmpp message when the build has completed. And does the needful site rebuilding. The xmppipe command I use is really the following script:

 #!/usr/bin/perl
 # pipes standard in to an xmpp message, sent to the JIDs on the commandline
 #
 # usage: bash$ `echo "message body" | xmppipe garen@tychoish.com
 #
 # code shamelessly stolen from:
 # http://stackoverflow.com/questions/170503/commandline-jabber-client/170564#170564

 use strict;
 use warnings;

 use Net::Jabber qw(Client);

 my $server = "tychoish.com";
 my $port = "5222";
 my $username = "bot";
 my $password = "";
 my $resource = "xmppipe";
 my @recipients = @ARGV;

 my $clnt = new Net::Jabber::Client;

 my $status = $clnt->Connect(hostname=>$server, port=>$port);

 if (!defined($status)) {
    die "Jabber connect error ($!)\n";
 }
 my @result = $clnt->AuthSend(username=>$username,
    password=>$password,
    resource=>$resource);

 if ($result[0] ne "ok") {
    die "Jabber auth error: @result\n";
 }

 my $body = '';
 while (<STDIN>) {
    $body .= $_;
 }
 chomp($body);

 foreach my $to (@recipients) {
     $clnt->MessageSend(to=>$to,
             subject=>"",
             body=>$body,
             type=>"chat",
             priority=>10);
 }

 $clnt->Disconnect();

Mark the above as executable and put it in your path somewhere. You’ll want to install the Net::Jabber Perl module, if you haven’t already.

The one final note. If you’re using a tool like gitosis to manage your git repositories, all of the hooks will be executed by the gitosis user. This means that this user will need to have write access the “build” copy of the repository and the public directory as well. You may be able to finesse this with the +s “switch uid” bit, or some clever use of the gitosis user group.

The End.

The Meaning of Work

2010-06-22T00:00:00-04:00

I’ve started to realize that, fundamentally, the questions I’m asking of the world and that I’m trying to address by learning more about technology, center on work and the meaning and process of working. Work lies at the intersection of the all the things that I seem to revisit endlessly: interfaces, collaboration technology, cooperatives and economics institutions, and open source software development. I’m not sure if I’m interested in work because it’s the unifying theme of a bunch of different interests, or this is the base from which other interests spring.

I realize that this makes me an incredibly weird geek.

I was talking to caroline about our respective work environments, specifically about how we (and our coworkers) relocated (or didn’t) for our jobs, and I was chagrined to realize that this novel that I’ve been working at (or not,) for way too long at this point spends some time revolving around these questions:

How does being stuck in a single place and time constrain one agency to effect the world around them?
What does labor look like in a mostly/quasi post-scarcity world?

Perhaps the most worrying thing about this project is that I started writing this story in late August of 2008. This was of course before the American/Financial Services economic crash that got me blogging and really thinking about issues outside of technology.

It’s interesting, and perhaps outside the scope of this post, but I think it’s interesting how since graduating from college, my “research” interests as they were, all work them into fiction (intentionally or otherwise.) I suppose I haven’t written fiction about Free Software/open source, exactly, but I think there’s a good enough reason for that.¹

I’m left with two realizations. First, that this novel has been sitting on my plate for far too long, and there’s no reason why I can’t write the last 10/20 thousand words in the next few months and be done with the sucker. Second, I’m interested in thinking about how “being an academic” (or not) affects the way I (we?) approach learning more about the world and the process/rigor that I bring to those projects.

But we’ll get to that later, I have writing to do.

I write fiction as open source, in a lot of ways, so it doesn’t seem too important to put it in the story as well.
↩

There's Always Something New to Learn

2010-06-17T00:00:00-04:00

Now that I’m fairly confident in my ability to do basic Linux systems administration tasks: manage web and email servers, maintain most Linux systems, convince desktop systems that they really do want to work the way they’re supposed to, I’m embarking on a new learning process. I’ve been playing around with “real” virtualization on my desktop, and I’ve been reading a bunch about systems administration topics that are generally beyond the scope of what I’ve dealt with until now. Here is a selection of the projects I’m playing with:

Getting a working xen setup at home. This requires, learning a bit more about building working operating systems, and also in the not to distant future buying a new (server) computer.
Installing xen on the laptop, because it’ll support it, I have the resources to make it go, and it’ll be awesome.
Learning everything I can about LVM, which is a new (to me) way of managing partitions and disk images, that makes backups, disk snapshots, and other awesomeness much easier. It means, some system migration stuff that I have yet to tinker with, as none of my systems currently support LVM.
Doing package development for Arch Linux, because I think that’s probably within the scope of my ability, because I think it would add to my skill set, and because I appreciate the community, and I want to be able to give back. Also I should spend some time editing the wiki, because I’m really lazy with that.

I guess the overriding lesson of all these projects is a more firm grasp of how incredibly awesome, powerful, and frankly stable Arch Linux is (or can be.) I mean there are flaws of course, but given how I use systems, I’ve yet to run into something show stopping. That’s pretty cool.

The Old Projects Project

2010-06-16T00:00:00-04:00

Before a road trip a, by now, a couple of months ago, I installed a copy of nginx on my laptop on the hope of doing some web development and working on other projects when I was in the car. For the uninitiated (you mean you don’t all write technical documentation for web developers and systems administrators?!?) nginx is an incredibly powerful web server. As of June 11th, foucualt the server that hosts the Cyborg Institute and tychoish.

This is, almost always, I think, a loosing proposition.

I never get any sort of substantial (or insubstantial) work done during my road trips up-and-down the north east corridor. Not that that’s a bad thing, but I also expect that there’ll be more awake-time when I’m not driving or gossiping.

And there never is.

So the web server sat unused for a long time on my laptop, but recently I’ve been playing with it a bit and I’ve finally gotten a number of cool things set up. I have a local “git web” instance which makes it easier to track progress on local and private projects that are stored in git. Perhaps more importantly, I have set up quick local ikiwiki instances for a number of projects. They’re easy to configure, quick to setup, and while I suppose I could hack something together in nifty for myself, there’s something nifty about being able to take an alternate view of some content and also being able to really preview changes to you work before publishing them.

Also, and the real reason for this post, is that by virtue of this development, I have revisited a few projects that had been lingering in the home directory of my computer for far too long. Which has been a powerful and useful exercise.

By which I mean, it’s been painful.

Besides “the novel,” which has been the lingering and dragging front burner project for a year, there are a number of quasi-serial stories that have lingered in some state of incompleteness for a couple of years now. I’m kind of amazed both at how foreign these stories seem to me both in terms of the style (good to know that I’m a better writer than I was a few years ago,) and also how quickly I can fall right back into the story and tell you every little thing about the world, situation, and moment where I left off.

The mind is, indeed, an amazing thing.

Where my strategy for the past year has been to “plow through and finish the novel,” I think my tactic this summer will be to move all of my projects forward in some way. Small daily writing goals for the novel, combined with somewhat less regular (but more specific) goals with regards to other projects. In the next two months I want to have a fairly active and varied writing schedule worked out that isn’t based around the monthly (or so) weekend binges that I’ve been using for most of the last year.

That’s the plan at any rate.

Jekyll and Automation

2010-06-08T00:00:00-04:00

As this blog ambles forward, albeit haltingly, I find that the process of generating the site has become a much more complicated proposition. I suppose that’s the price of success, or at least the price of verbosity.

Here’s the problem: I really cannot abide by dynamically generated publication systems: there are more things that can go wrong, they can be somewhat inflexible, they don’t always scale very well, and it seems like horrible overkill for what I do. At the same time, I have a huge quantity of static content in this site, and it needs to be generated and managed in some way. It’s an evolving problem, and perhaps one that isn’t of great specific interest to the blog, but I’ve learned some things in the process, and I think it’s worthwhile to do a little bit of rehashing and extrapolating.

The fundamental problem is that the rebuilding-tychoish.com-job takes a long time to rebuild. This is mostly a result of the time it takes to convert the Markdown text to HTML. It’s a couple of minutes for the full build. There are a couple of solutions. The first would be to pass the build script some information about when files were modified and then have it only rebuild those files. This is effective but ends up being complicated: version control systems don’t tend to version mtime and importantly there are pages in the site–like archives–which can become unstuck without some sort of metadata cache between builds. The second solution is to provide very limited automatically generated archives and only regenerate the last 100 or so posts, and supplement the limited archive with more manual archives. That’s what I’ve chosen to do.

The problem is that even the last 100 or so entries takes a dozen seconds or more to regenerate. This might not seem like a lot to you, but the truth that at an interactive terminal, 10-20 seconds feels interminable. So while I’ve spent a lot of time recently trying to fix the underlying problem–the time that it took to regenerate the html–when I realized that the problem wasn’t really that the rebuilds took forever, it was that I had to wait for them to finish. The solution: background the task and send messages to my IM client when the rebuild completed.

The lesson: don’t optimize anything that you don’t have to optimize, and if it annoys you, find a better way to ignore it.

At the same time I’ve purchased a new domain, and I would kind of like to be able to publish something more or less instantly, without hacking on it like crazy. But I’m an edge case. I wish there were a static site generator, like my beloved jekyll that provided great flexibility, and generated static content, in a smart and efficient manner. Most of these site compilers, however, are crude tools with very little logic for smart rebuilding: and really, given the profiles of most sites that they are used to build: this makes total sense.

I realize that this post comes off as pretty complaining, and even so, I’m firmly of the opinion that this way of producing content for the web is the most sane method that exists. I’ve been talking with a friend for a little while about developing a way to build websites and we’ve more or less come upon a similar model. Even my day job project uses a system that runs on the same premise.

Since I started writing this post, I’ve even taken this one step further. In the beginning I had to watch the process build. Then I basically kicked off the build process and sent it to the background and had it send me a message when it was done. Now, I have rebuilds scheduled in cron, so that the site does an automatic rebuild (the long process) a few times a day, and quick rebuilds a few times an hour.

Is this less efficient in the long run? Without a doubt. But processors cycles are cheap, and the builds are only long in the subjective sense. In the end I’d rather not even think that builds are going on, and let the software do all of the thinking and worrying.

Organize Your Thoughts More Betterly

2010-06-02T00:00:00-04:00

I’ve been working with a reader and friend on a project to build a tool for managing information for humanities scholars and others who deal with textual data, and I’ve been thinking about the problem of information management a bit more seriously. Unlike numerical, or more easily categorized information data, how to take a bunch of textual information–either of your own production or a library of your own collection–is far from a solved problem.

The technical limitation–from a pragmatic perspective–is that you need to have an understanding not only of the specific tasks in front of you, but a grasp of the entire collection of information you work with in order to effectively organize, manage, and use the texts as an aggregate.

“But wait,” you say. “Google solved this problem a long time ago, you don’t need a deterministic information management tool, you need to brute force the problem with enough raw data, some clever algorithms, and search tools,” you explain. And on some level you’d be right. The problem is of course, you can’t create knowledge with Google.

Google doesn’t give us the ability to discover information that’s new, or powerful. Google works best when we know exactly what we’re looking for, the top results in Google are most likely to be the resources that the most people know and are familiar. Google’s good, and useful and a wonderful tool that more people should probably use but Google cannot lead you into novel territory.

Which brings us back to local information management tools. When you can collect, organize, and manipulate data in your own library you can draw novel conclusions, When the information is well organized, and you can survey a collection in useful and meaningful ways, you can see holes and collect more, you can search tactically, and within subsets of articles to provide. I’ve been talking for more than a year about the utility of curation in the creation of value on-line. and fundamentally I think the same holds true for personal information collections.

Which brings us back to the ways we organize information. And my firm conclusion that we don’t have a really good way of organizing information. Everything that I’m aware of either relies on search, and therefore only allows us to find what we already know we’re looking for, or requires us to understand our final conclusions during the preliminary phase of our investigations.

The solution to this problem is thus two fold: First, we need tools that allow us to work with and organize the data for our projects, full stop. Wiki’s, never ending text files, don’t really address all of the different ways we need to work with and organize information. Secondly we need tool tools that are tailored to the way researchers who deal in text work with information from collection and processing to quoting and citation, rather than focusing on the end stage of this process. These tools should allow our conceptual framework for organizing information to evolve as the project evolves.

I’m not sure what that looks like for sure, but I’d like to find out. If you’re interested, do help us think about this!

(Also, see this post regarding the current state of the Cyborg Institute.)

The Schedule

2010-05-26T00:00:00-04:00

Wow. Hello blog.

I’m pretty busy. It even seems sort of cliche to complain about such things on ones blog, but I think being busy has coincided with a somewhat larger reevaluation of nearly everything.

Wait, no. I’m not quitting blogging.

I’m actually really proud of the Knowing Mars launch, and it feels really good to have that project “done,” even if I think it needs a major revision, and I have a lot more fiction on my plate that I don’t want to just “let go” like that.

I’m also somewhat displeased with the kind of blog posts that I’ve been writing recently. It seems that I’ve been writing about some basic ideas: my disdain for the way the web functions as a user interface, some general work flow topics, some basic cyber-culture topics, and half way through most of these blog posts I mostly loose interest, and I suspect you have as well.

I’ve had a post in my “write this soon list,” about digging in deeper and striving for a more rich engagement with the topics I try and cover here, and I’ve pretty much failed with that. In any case, this post was supposed to be more about the things that are on my schedule:

I’ve been doing a lot offline these past few weeks. It’s May and that means it’s Morris Dancing season. I seem to have joined an interesting phenomena called “Maple Morris,” (more reflection on that when I’ve processed a bit more,) the usual Mayday festivities, Midwest Morris Ale. And then there are a bunch of singing conventions, which are a great deal of fun and fulfilling, and then there are contra dancing things, but none of these things transmit to quiet weekends alone writing. Or even quiet evening around writing. At least very often. Some highlights of the recent past and near future:

Maple Morris; A my-generation Morris dancing event, last weekend in Boston. A bunch of Morris dancers in my general age rage got together to dance some really challenging dances and to sing great songs. I was totally overwhelmed.
The Midwest Morris Ale; My regular annual Morris dancing ale. This is my 9th consecutive ale (and my 10th anniversary of dancing Morris.)
Since last September, I’ve gone to an all- or mulit- day Sacred Harp singing convention most months, since last September, and there’s one on my calender every month between now and this September.
I’m going to ”Youth Dance Weekend” in Vermont in September, which I’ve never been to, but I think it’ll be a a great deal of fun, and I’m very much looking forward to it. I’ve not been contra dancing as much, but that’s not a huge problem for me.
I’m moving to Philadelphia in the summer, which means a drastically longer commute, but an easier to orchestrate social life, and a better work/life balance. This means apartment hunting and all that jazz.

While this means less writing time and time for taking care of my own projects, it doesn’t mean that I don’t have any writing time. Sure writing takes time, but the largest challenge as a writer is in using the time I/we already have effectively, and getting the most out of those opportunities.

It also, I think, means finding a way to develop a writing (and blogging) habit that:

Doesn’t revolve around a fixed daily publication schedule. I still want to write essays, but I need to write essays when I have a compelling argument for an essay, rather than around the same core of ideas that I’ve been running around for the last year.
I need to be able to put the blog on the back burner while I focus on things like writing fiction, or hacking projects, or Cyborg Institute stuff. The blog is great, and I love writing the blog, but It’s far to easy for me to fall into a pattern where the blog becomes the project, rather than the journal in support of the project.
I need to organize my projects and tasks into clumps of work that are easier to manage in shorter periods of time. This is probably a reorganization problem that needs to mostly occur within my head.

So where does that leave us? I have a few posts piled up that I’ll parcel out over the next few weeks, though on the whole there will probably be less posting by me around here. I’m probably going to do more posts along the lines of “here’s what I’ve been up to, go read my work elsewhere.” There will be some guest posts and I’ve already begun working with some writers for that. Beyond that, I guess we’ll both be able to be surprised.

Knowing Mars, a Novella

2010-05-19T00:00:00-04:00

I don’t know about this. But here it goes, anyway.

I’m pleased to announce the complete publication of my novella ”Knowing Mars” on tychogaren.com.

“Knowing Mars” is an important story for me. I wrote it after I graduated from college, after I didn’t go to graduate school the first time, and in a lot of ways it was the project that got me started down the path of being a “real writer,” post-graduation. I’d written fiction before college, and mostly avoided writing fiction in college, and then right as I was finishing college I started writing stories again. It was strange for a while, but it was delightful to be able to tell stories and be so much better at it than I was the first time.

I find that this is fundamentally a recurring issue. When I started writing “Knowing Mars,” I felt like I was starting out light years ahead of what I had written four or five years before. Now, I feel like the stuff I’m working on now is light years ahead of “Knowing Mars.” This is probably an encouraging sign.

I suppose you’d like to know a bit more about the story. I’ll leave most of the details to the reading, but basically it’s a sort of superhero/cyberpunk story that explores themes related to diaspora, political organization, historical narrative, and gender. I don’t know if I’ve ever described it as such before.

The story is available in multiple formats. Each chapter is available in full HTML as part of tychogaren.com and in a plan Markdown formatted plain text. Furthermore the complete text of the novella is available in both of these formats, and a simple un-styled HTML version that should be ideal for conversion to various electronic reading platforms. If you want to read the novella but would find another format easier to process, talk to me about it and I’ll get something pulled together for you.

All “full html” versions of the text have comments enabled using the same system as the blog. I look forward to your comments. Thanks for reading, and stay tuned for more fiction.

Onward and Upward!