A friend asked me a question (several weeks ago by publication) on a technical topic and I spent most of the next few days writing a missive on database administration strategy. That seemed like a normal response. I was delighted to find that: I liked the voice, I enjoyed writing the longer document, and there are a dozen or so other related topics that I wanted to explore. So, apparently, I’m writing a book. This is exactly what I need: more projects. Not.

But it’s a good thing: I find the writing inspiring and invigorating. I have a perspective and collection of knowledge that hasn’t been collected and presented in a single place. I like long form writing. The larger piece might also be a good contribution to my portfolio (such as it is.)

I think this kind of writing suits my attention span.

This has left me without a lot of spare time for blogging, and (as I’m prone to do every so often,) rethinking the future of my efforts on tychoish.com and as a blogger. This is boring for all of you, but I’ll give some higher level stuff here and we can follow up in comments:

• Blogging is fun, and even though I’ve not been posting regularly, I’m always writing blog posts. Sometimes I find myself writing posts in emails to friends, but I’m never really going to stop writing blog posts.

• The general explosion of blog publishing that we saw a few years ago has declined. Audience fragmentation happened, readership got entrenched. I feel like I weathered the storm pretty well and I’m really happy with the site and readers I have, but I’m also pretty confident that blogging isn’t going to be the means by which I “level up.”¹

• eBooks have finally happened. For the last decade most people have been saying that ebooks are great for reference material (given search-ability,) and for providing an introduction to a text that people will eventually buy in a paper edition. That may be true, but I think it’s changing rapidly, and with kindles and tablets and smart-phones, I think eBooks have effectively won, such as it is.

  In another ten years, perhaps, we’ll just call them books.

• I’m pretty clear that keeping a blog, and perhaps most of the writing I do in my spare time is for my own enjoyment and betterment and helps to develop a personal portfolio and account of my work. I have no (real) interest in using my writing on tychoish.com or any other side that I maintain, as a way of supporting myself to any greater or lesser extent.

I want to be in the business of writing things and working with technology and ideas and people, not the business of publishing. While the line is not always clear between “writing projects that you publish yourself online,” and “new media publisher,” I want to stay away from the later as much as possible.

So I think this means that most of my “tychoish,” writing time will go to writing this book project, and to fiction, and once my blog post backlog is fully depleted (heh,) most of my postings will either be announcements/progress-reports or a bunch of shorter more off-the-cuff notes.

Here’s hoping at least.

This post is a follow up to my earlier post on diversity and representation In short, while I think it’s great that we’re beginning to talk and write about race and representation in our fiction and field, I think we¹ need to expand our analysis of whiteness.

Whiteness in Science Fiction

I’m still working on figuring out what this means, and I’m sorry that I haven’t developed my thinking sufficiently to be more clear on this. In light of that here are a collection of my thoughts on representation:

• Whiteness is multiple and I think it’s possible (and important) to depict whiteness and white characters critically and without recapitulating normalization. At the same time, it’s important to avoid falling victim to a lot of the normalization to which uncritical representations of racial diversity often fall pray.
• The theory around race and representation must deal with issues around assimilation. More diversity is useful, but to move forward on issues of representation, the field needs to better understand the process of assimilation. I want to see stories that help us unpack assimilation.
• Whiteness is complex and a major problem with stories that “don’t do race well,” is not just that the characters aren’t explicitly of color, but that whiteness isn’t portrayed very well. This is part of the struggle of privilege, but not only does science fiction need to be better about diversity and representation of non-white characters, but we the thinking on whiteness needs to continue to evolve apace.

Diversity and Quotas

Discussions about diversity and representation in fiction often lead the under-informed to ask “So what, do you want to impose some sort of quota system? Does that mean diversity is more important than quality?”

The answer is almost always no.

I’d also like to point out that this is one of those cases where whiteness and systematic bias conspire to define “quality,” in unuseful ways. But this is another argument for another time.

The canonical answer is: there’s a great deal of amazing work written by people of color and a lot of great fiction that incorporates and addresses the experiences of people of color. This is great, and if we’ve learned anything in the last couple of years, it’s that if you look for this work it’s there. The real challenge revolves around cultivating that work so that there’s more of it, and promoting² that work so that there’s a large audience.

On Outer Alliance Podcast #8, David Levine talked about having worked as a technical writer for some 15 years and then said something to the effect of “It’s a point of great personal pride that I’ve never put a bulleted list in a piece of fiction.”

I laughed out loud. Perhaps frightening a woman walking her dog nearby.

In most ways, the kind of writing that I do for work, API references, tutorials, administration overviews, best-practice descriptions, is very different from the kinds of things I write away from work, or at least I like to think so.

The truth is that I’ve learned a bunch about writing and about communicating in general from writing documentation. While my “professional background,” doesn’t include formal technological training, I definitely “broke in” because I was familiar with technology and could write, rather than being a particularly skilled or trained writer. Any more (just 2.5 years on,) I think the inverse is more true, but that’s conjecture.

Technical writing has definitely shaped the evolution of my taste: a couple years ago, I found myself most drawn to complex tightly constructed prose in fiction. These days I mostly go for sparse clear concise prose that isn’t particularly ornamented. Perhaps it’s only really possible to tune the internal editor for one kind of style at a time.

Having said that, I will confess to feeling--and resisting--the urge to put a bulleted list or some other structured convention of software manuals in fiction.

It’s the little things, really.

When I bought what is now my primary laptop, I had intended to use the extra flexibility to learn the prevailing (industrial-grade) virtualization technology. While that project would have been edifying on its own, I also hoped to use the extra flexibility to some more consistent testing and development work.

This project spurned a xen laptop project, but the truth is that Xen is incredibly difficult to get working, and eventually the “new laptop” just became the “every day laptop,” and I let go of the laptop Xen project. In fact, until very recently I’d pretty much given up on doing virtualization things entirely, but for various reasons beyond the scope of this post I’ve been inspired to begin tinkering with virtualization solutions again.

As a matter of course, I found myself trying KVM in a serious way for the first time. This experience both generated a new list of annoyances and reminded me about all the things I didn’t like about Xen. I’ve collected these annoyances and thoughts into the following post. I hope that these thoughts will be helpful for people thinking about virtualization pragmatically, and also help identify some of the larger to pain points with the current solution.

Xen Hardships: It’s all about the Kernel

Xen is, without a doubt, the more elegant solution from a design perspective and it has a history of being the more robust and usable tool. Performance is great, Xen hosts can have up-times in excess of a year or two.

The problem is that dom0 support has, for the past 2-3 years, been in shambles, and the situation isn’t improving very rapidly. For years, the only way to run a Xen box was to use an ancient kernel with a set of patches that was frightening, or a more recent kernel with ancient patches forward ported. Or you could use cutting edge kernel builds, with reasonably unstable Xen support.

A mess in other words.

Now that Debian Squeeze (6.0) has a pv-ops dom0 kernel, things might look up, but other than that kernel (which I’ve not had any success with, but that may be me,) basically the only way to run Xen is to pay Citrix¹ or build your own kernel from scratch, again results will be mixed (particularly given the non-existent documentation,) maintenance costs are high, and a lot of energy will be duplicated.

What to do? Write documentation and work with the distributions so that if someone says “I want to try using Xen,” they’ll be able to get something that works.

KVM Struggles: It’s all about the User Experience

The great thing about KVM is that it just works. “sudo modprobe kvm kvm-intel” is basically the only thing between most people and a KVM host. No reboot required. To be completely frank, the prospect of doing industrial-scale virtualization on-top of nothing but the Linux kernel and with a wild module in it, gives me the willies is inelegant as hell. For now, it’s pretty much the best we have.

The problem is that it really only half works, which is to say that while you can have hypervisor functionality and a booted virtual machine, with a few commands, it’s not incredibly functional in practical systems. There aren’t really good management tools, and getting even basic networking configured off the bat, and qemu as the “front end” for KVM leaves me writhing in anger and frustration.²

Xen is also subject to these concerns, particularly around netowrking. At the same time, Xen’s basic administrative tools make more sense, and domU’s can be configured outside of interminable non-paradigmatic command line switches.

The core of this problem is that KVM isn’t very Unix-like, and it’s a problem that is rooted in it’s core and pervades the entire tool, and it’s probably rooted in the history of its development.

What to do? First, KVM does a wretched job of anticipating actual real-world use cases, and it needs to do better at that. For instances it sets up networking in a way that’s pretty much only good for software testing and GUI interfaces but sticking the Kernel on the inside of the VM makes it horrible for Kernel testing. Sort out the use cases, and there ought to be associated tooling that makes common networking configurations easy.

Second, KVM needs to at least pretend to be Unix-like. I want config files with sane configurations, and I want otherwise mountable disk images that can be easily mounted by the host.

Easy right?

This is the story of how I occasionally realize I exist on the continuum of “programmers,” rather than just being an eccentric sort of writer type.

::: {.contents} :::

Evidence

download-mail

I have this somewhat peculiar method of downloading email that I think works great. A few weeks ago, however, I was trying to compress things in “hot storage,” and realized that I had a problem.

For a year or so, I had been automating commits to the git repository that held all my mail. In order to effectively archive and compress some mail, I needed to do some serious rebasing to not only remove a bunch of messages from the current repository but also pull that content from the history and flatten the history somewhat.

The problem was that I had 50,000 commits and there’s simply no effective way to rebase that many commits in a reasonable amount of time, particularly given I/O limitations. So I gave up, started from (relative) scratch, and rewrote the scripts to be a little bit more smart… You know in an afternoon.

See the revised code here: download mail

ikiwiki-tasklist

I’ve written about this before in my post on my new personal organization stuff, but it’s no great announcement that I’m moving away from working in emacs’ org-mode and doing more work with ikiwiki and some hand-rolled scripts. I think org-mode is great, it just ended up getting in my way a bit and I think I can get more of what I need to get done in other ways.

I have learned a great deal from org-mode. I made the biggest leap away from org-mode when I wrote ikiwiki tasklist, which does all of the things I had been using org-mode’s agenda for. It’s not a complicated at all: look in some files for some lines that begin with specific strings and put them into a page that is the perfect task list.

See the code here: ikiwiki tasklist.

Common Lisp Weenie

“What Window Manager is that,” he asked.

“StumpWM, it’s written in Common Lisp,” I said, launching into a 30 second pitch for Stump.

My pitch about stump is pretty basic: the Common Lisp interface allows you to evaluate code during run-time without restarting the window manager or loosing state; it’s functionally like screen, which is very intuitive for window management; and it has emacs-like key-bindings, which I think work pretty well.

“So you’re a Common Lisp programmer?”

“Well not really, I mean, I know enough to get by.”

“Right.”

“Right.”

Conclusion

In several (technical writing) job interviews recently, people asked me about my programming experience, and my answer varied a lot.

I know how computer programs work, I know how people write computer programs, I understand how software testing and debugging works, I understand the kinds of designs that lead to good programs and the kinds that lead to bad software. I don’t write code--really--but I can sort of hack things together in shell scripts when I need to.

The answer to the question, these days, is “I’m a programmer in the way that most people are writers: most people are comfortable writing a quick email or a short blurb, but get stuck and have trouble really writing longer or more complicated kinds of text. Reasonably capable but not skilled.”

The above code examples work: they do what I need them to do, and particularly in the case of the mail script, they work much better than the previous iteration. I need to do more work, and I feel like I’m reaching the boundaries of what can be comfortably done in shell scripting. My next big programming project is to go through these two scripts and port them to Python and see if I can add just a little bit of additional functionality in the process.

I’m sure I’ll report to you on this as my work progresses.

I added a few lines to a script that I use to build my task list, and for the first time ever, I opened a file with code in it, added a feature, tested it, and it worked. Here’s the code with enough context so it makes sense (explained later if you don’t want to spend the time parsing it:)

ARG=`echo "$@" | sed -r 's/\s*\-[c|p|s]\s*//g'`
WIKI_DIR="`echo $ARG | cut -d " " -f 1`"
if [ "`echo $ARG | cut -d " " -f 2 | grep -c /`" = 1 ]; then
   TODO_PAGE="`echo $ARG | cut -d " " -f 2`"
elif [ "`echo $ARG | cut -d " " -f 2 | grep -c $EXT`" = 1 ]; then
   TODO_PAGE="$WIKI_DIR/`echo $ARG | cut -d " " -f 2`"
else
   TODO_PAGE="$WIKI_DIR/`echo $ARG | cut -d " " -f 2`.$EXT"
fi

This is from the section of the script that processes the arguments and options on the command line. Previously, commands were issued such that:

ikiwiki-tasklist [-c -p -s] [DIR_TO_CRAWL] [OUTPUT TODO FILE NAME]

My goal with the options was to have something that “felt like” a normal command with option switches and had a lot of flexibility. The two fields that followed: however, I didn’t provide as much initial flexibility. The directory to crawl for tasks (i.e. “[DIR_TO_CRAWL]”) was specified the way it is now, but the output file was 1) assumed to have an extension specified in a variable at the top of the script, 2) automatically placed the output file in the top level of the destination directory.

It worked pretty well, but with the advent of a new job I realized that I needed some compartmentalization. I needed to fully use the tasklist system for personal and professional tasks without getting one set of items mixed in with the other. Being able to have better control of the output is key to having full control over this.

The modification detects if the output file looks like a path rather than a file name. If it’s senses a path, it creates the task list in the path specified, with no added extension. If a file name specifies the extension, then you won’t get “.ext.ext” files. And the original behavior is preserved.

I’m a hacker by inclination: I take code that I find and figure out how to use it. Sometimes I end up writing or writing code, but I’m not really a programmer. My own code, at least until recently has tended to be somewhat haphazard and until now (more or less) I’ve not felt like I could write code from scratch that was worth maintaining and enhancing in any meaningful way.

Apparently some of that’s changed.

I’ve made a few additional changes to the scripts, but most of these feel more trivial and can be described as “I learned how to write slightly tighter shell scripts. so if you’re using it you might want to update: the ikiwiki tasklist page is up to date.

I had an item on my list of blog posts to write for a couple of years to write something reflecting on “RaceFail,” and finally a gave up, because I didn’t want to write a book, I didn’t know what to say, and I was more interested in the actual discourse itself than finding the “side of right,” in a conversation that was both way too simple and way too complex all at once.

So rather than reboot the conversation, which has ended in some senses and continues on in others, I want to start writing a bit here about race and representation in fiction, but also discussing the way that conversations transpire online. Here’s part one. I’ll figure out some way to index them all together once they’re posted and assembled.

I wrote this scene a while back where a character who grew up on a small¹ outpost visits a space ship. Given relativistic space travel, from the character’s perspective, the crew of the space ship are 750 years old or so, despite being in their subjective early forties. That means the character’s 31st-great-grandparents (roughly) were cousins of the people he’s looking at.

He notices a few things: the people on the ship are all taller than he is and also taller than everyone from the outpost. He also notices that there’s more more skin tone variation amongst the people on the ship than there is among the people in the outpost.

There are a bunches of problems with this story. Including the fact that its not finished and that there are parts of the execution that I think need a lot work. But this part, I quite like. For this story (and I think in general,) I’ve drawn the following conclusion:

• Race is temporally constrained. We understand racial difference and our own racial experiences in terms of our current reality. This changes.
• The aspects of race which are the result of lineage (skin color, bone structure,) are likely to change over time as lineages continue. We can assume that these kinds of changes will be pronounced in smaller populations over longer periods of time.

To a large extent the tension between the “outpost people” and the “ship people” is the core of the conflict in this story. I’ve been thinking in this story about the impact of colonialism (and race as a result) on societies and political outlook. It’s almost certainly not perfect, but I enjoy the possibilities, the story has its moments, and I’m finding the theory building productive.

I’m circling around a point: in-story diversity, particularly, diversity that reflects late 20th/early 21st century notions of difference alone cannot further thought race and racism. In other words, diversity is not criticism. There are many ways to productively further the discussion of difference in (genre) fiction, lets not stop with representation.

I’ll be writing more about this in the future. Comments are very welcome!

I was having a conversation with a (now former) coworker (a while ago) about the future of shared file systems, unstructured organization and management, and access control. What follows are a collection of notes and thoughts on the subject that have stuck with me.

Let’s start with some general assumptions, premises, ideas:

• File system hierarchies are dead or dying. To have a useful file system hierarchy the following qualities are essential:

• Every piece of data needs to belong in one location and only one location.

• Every container (e.g. directory or folder) needs to hold at least two objects.

• Hierarchy depth ought to be minimized. Every system can use two levels. After the second level, each additional level should only be added a very large number of additional objects are added to the system. If you have 3 functional levels and less than 1000 objects, you might be in trouble.

  As you might imagine, this is very difficult to achieve, and the difficulty is compounded by huge amounts of legacy systems, and the fact that “good enough is good enough,” particularly given that file organization is secondary to most people’s core work.

  While there are right ways to build hierarchical structure for file system data, less structure is better than more structure, and I think that groups will tend toward less over time.

• Access control is a lost cause. Legacy data and legacy practices will keep complex ACL-based systems for access control in place for a long time, but I think it’s pretty clear that for any sort of complex system, access control isn’t an effective paradigm. In some ways, access control is the last really good use of file system hierarchies. Which is to say, by now the main use of strong containers (as opposed to tags) is access control.

  I don’t think that “enterprise content management”-style tools are there, yet. I suspect that the eventual solution to “how do I control access to content” will either: be based on a an cryptography key system which will control access and file integrity, or there will be a class of application, a la ECMS, with some sort of more advanced abstracted file system interface that’s actually use-able.

I’m betting on encryption.

• Tagging and search are the ways forward. In many cases, the location of files in hierarchy help determine the contents of those files. If there are no hierarchies then you need something more useful and more flexible to provide this level of insight.

• Great search is a necessity. Luckily it’s also easy. Apache Solr/Lucene, Xapian, and hell Google Search Appliances make great search really easy.

• Some sort of tagging system. In general, only administrators should be able to create tags, and I think single tag-per object (i.e. categories) versus multiple tags per object should be configurable on a collection-by-collection.

  Tag systems would be great for creating virtualized file system interfaces, obviating the need for user-facing links, and leveraging existing usage patterns and interfaces. It’s theoretically possible to hang access control off of tag systems but that’s significantly more complicated.

  One of the biggest challenges with tag systems is avoiding recapitulating the problems with hierarchical organization.

The most difficult (and most interesting!) problem in this space is probably the access control problems. The organizational practices will vary a lot and there aren’t right and wrong answers. This isn’t true in the access control space.

Using public key infrastructure to encrypt data may be an effective access control method. It’s hard replicate contemporary access control in encryption schemes. Replicating these schemes may not be desirable either. Here are some ideas:

• By default all files will be encrypted such that only the creator can read it. All data can then be “world readable,” as far as the storage medium and underlying file systems are concerned.

• The creator can choose to re-encrypt objects such that other users and groups of users can access the data. For organizations this might mean a tightly controlled central certificate authority-based system. For the public internet, this will either mean a lot of duplicated encrypted data, or a lot of key chains.

• We’ll need to give up on using public keys as a method of identity testing and verification. Key signing is cool, but at present it’s complex, difficult to administer, and presents a significant barrier to entry. Keys need to be revocable, particularly group keys within organizations.

  For the public internet, a some sort of social capital or network analysis based certification system will probably emerge to supplement for strict-web-of-trust based identity testing.

• If all data is sufficiently encrypted, VPNs become obsolete, at least as methods for securing file repositories. Network security is less of a concern when content is actually secure. Encryption overhead, for processing isn’t a serious concern on contemporary hardware.

Thoughts?