Wiki Fiction and Critical Futures

I'm starting preliminary development on a wiki fiction project that will eventually take over the criticalfutures.com domain. This post is a discussion of that history, my idea, and what I hope to accomplish.

My friend Julia and I have been corresponding on topics related to the future of publishing and genre fiction for a few weeks. When the topic turned to wikis, I spouted off the things I usually say about bootstrapping wiki participation (it's hard and pretty lonely,) then I had an idea.

I read wikis, mostly wikipeida, a lot. For fun. I'm sure a lot of people do this, as "getting lost in wikipedia," is a thing that happens. You say, "I'm interested in public transit in Iran," and you get lost clicking through various pages related to rapid transit systems in the middle east, and then an hour or two is past and you really ought to finish that blog post. The same thing happens on the c2 wiki for me.

While I wish I were less compulsive about it, reading wikis is a pleasurable reading experience, and since the format seems the web, why not run with it? The question becomes: why are we spending so much time figuring out the most ideal way to publish novels and short stories--forms that developed with the physicality of the book--in the digital age?

To be fair, I think there's a place for digital distribution of paper-centric forms (periodicals and monographs,) but I doubt that in 50 years "digital fiction," will mean eBook editions of novels. People have been making a similar point for some time about video games for a while. Interactive fiction is definitely a part of digital fiction, but I don't think it's the full story.

Meanwhile back at the point...

Here's the idea. We use wiki software to construct a website that is written as a light-hearted encyclopedia. In the vein of TV Tropes meets Wikipedia except with fictional content. But there needs to be more than just page after page of exposition and condensed blather: my current plan is to have a "dialogue" section, which will be bits of dialogue and scenes published with some contextual metadata (when it happened, who was present, where it happened.) The dialogues can then be linked to as quasi-citations in the more conventional expository wiki pages.

So basically I'm proposing a couple of things here. First, I want to splitting up all content into small self contained pages. This makes it better for multiple people to edit, because editing and writing can happen in a more parallel manner, and you don't need to agree to an outline, or write things in any sort of sequence. Second, shorter pages with more segmented content is easier to read for the attention limited.

Having said that, I'm not sure that collaborative, for all that will editing is really the way to go. The truth is that so few people edit wikis relative to the number of people who could edit wikis, that you might be better off having some sort of more select editorial community, just in terms of establishing buy-in from contributors and avoiding diffusion of responsibility. I'm undecided.

Along a similar line of thought, I'm considering releasing release updates and new content on a regular basis (e.g. bi-weekly or monthly?) rather than every time an edit is made. This will require some sort of closed-development process. At the same time new wiki projects often fail because there's little incentive to return to a wiki to "see what's changed. Blogs, contrast are good at securing return visits.

Thoughts? Anyone interested in being on the editorial board?

The Meaning of Work

I've started to realize that, fundamentally, the questions I'm asking of the world and that I'm trying to address by learning more about technology, center on work and the meaning and process of working. Work lies at the intersection of the all the things that I seem to revisit endlessly: interfaces, collaboration technology, cooperatives and economics institutions, and open source software development. I'm not sure if I'm interested in work because it's the unifying theme of a bunch of different interests, or this is the base from which other interests spring.

I realize that this makes me an incredibly weird geek.

I was talking to caroline about our respective work environments, specifically about how we (and our coworkers) relocated (or didn't) for our jobs, and I was chagrined to realize that this novel that I've been working at (or not,) for way too long at this point spends some time revolving around these questions:

  • How does being stuck in a single place and time constrain one agency to effect the world around them?
  • What does labor look like in a mostly/quasi post-scarcity world?

Perhaps the most worrying thing about this project is that I started writing this story in late August of 2008. This was of course before the American/Financial Services economic crash that got me blogging and really thinking about issues outside of technology.

It's interesting, and perhaps outside the scope of this post, but I think it's interesting how since graduating from college, my "research" interests as they were, all work them into fiction (intentionally or otherwise.) I suppose I haven't written fiction about Free Software/open source, exactly, but I think there's a good enough reason for that. [1]

I'm left with two realizations. First, that this novel has been sitting on my plate for far too long, and there's no reason why I can't write the last 10/20 thousand words in the next few months and be done with the sucker. Second, I'm interested in thinking about how "being an academic" (or not) affects the way I (we?) approach learning more about the world and the process/rigor that I bring to those projects.

But we'll get to that later, I have writing to do.

[1]I write fiction as open source, in a lot of ways, so it doesn't seem too important to put it in the story as well.

Jekyll and Automation

As this blog ambles forward, albeit haltingly, I find that the process of generating the site has become a much more complicated proposition. I suppose that's the price of success, or at least the price of verbosity.

Here's the problem: I really cannot abide by dynamically generated publication systems: there are more things that can go wrong, they can be somewhat inflexible, they don't always scale very well, and it seems like horrible overkill for what I do. At the same time, I have a huge quantity of static content in this site, and it needs to be generated and managed in some way. It's an evolving problem, and perhaps one that isn't of great specific interest to the blog, but I've learned some things in the process, and I think it's worthwhile to do a little bit of rehashing and extrapolating.

The fundamental problem is that the rebuilding-tychoish.com-job takes a long time to rebuild. This is mostly a result of the time it takes to convert the Markdown text to HTML. It's a couple of minutes for the full build. There are a couple of solutions. The first would be to pass the build script some information about when files were modified and then have it only rebuild those files. This is effective but ends up being complicated: version control systems don't tend to version mtime and importantly there are pages in the site--like archives--which can become unstuck without some sort of metadata cache between builds. The second solution is to provide very limited automatically generated archives and only regenerate the last 100 or so posts, and supplement the limited archive with more manual archives. That's what I've chosen to do.

The problem is that even the last 100 or so entries takes a dozen seconds or more to regenerate. This might not seem like a lot to you, but the truth that at an interactive terminal, 10-20 seconds feels interminable. So while I've spent a lot of time recently trying to fix the underlying problem--the time that it took to regenerate the html--when I realized that the problem wasn't really that the rebuilds took forever, it was that I had to wait for them to finish. The solution: background the task and send messages to my IM client when the rebuild completed.

The lesson: don't optimize anything that you don't have to optimize, and if it annoys you, find a better way to ignore it.

At the same time I've purchased a new domain, and I would kind of like to be able to publish something more or less instantly, without hacking on it like crazy. But I'm an edge case. I wish there were a static site generator, like my beloved jekyll that provided great flexibility, and generated static content, in a smart and efficient manner. Most of these site compilers, however, are crude tools with very little logic for smart rebuilding: and really, given the profiles of most sites that they are used to build: this makes total sense.


I realize that this post comes off as pretty complaining, and even so, I'm firmly of the opinion that this way of producing content for the web is the most sane method that exists. I've been talking with a friend for a little while about developing a way to build websites and we've more or less come upon a similar model. Even my day job project uses a system that runs on the same premise.

Since I started writing this post, I've even taken this one step further. In the beginning I had to watch the process build. Then I basically kicked off the build process and sent it to the background and had it send me a message when it was done. Now, I have rebuilds scheduled in cron, so that the site does an automatic rebuild (the long process) a few times a day, and quick rebuilds a few times an hour.

Is this less efficient in the long run? Without a doubt. But processors cycles are cheap, and the builds are only long in the subjective sense. In the end I'd rather not even think that builds are going on, and let the software do all of the thinking and worrying.

On Publishing

I've been thinking about publishing and the publishing industry of late. I'm sure some of it is related to my wanting of a kindle and my resulting thoughts on consolidation, and maybe some small measure of it has to do with the fact that sometimes it easier to think about publishing and the future of publishing than it is to think about ones own creative projects. So be it.

First, "what is there to think about?" you ask? Well, lots of things: I've written about wanting a kindle, and some thoughts about consolidation, and finally some thoughts on digital publishing More recently I've been thinking more about the "work" of publishing and content creation, apart from the changing business models and technological context.

Publishers (of any kind, and their editorial departments), by contemporary convention are responsible for reading through the slush and figuring what's good and what not. Ideally publishers are the stewards of taste, and the people who figure out whats "good" and what people want to read. On some fundamental level, publishers are curators. The second main function of publishers are as the provider and organizer around services. Publishers contract with copy editors, with design and layout people, they get the cover art, they do promotional work, and the million other things it takes to turn a manuscript into a book.

As the traditional publishing model has... deteriorated, I think a lot of people have been interested in figuring out "what happens next?" myself as much as anyone. Having said that, the way in which the traditional publishing model has deteriorated has shaped how we think about what comes next. This makes sense of course, but I want to challenge myself to think about things more broadly (and you, dear friends as well, but I'm sure you've already figured this one out.)

I mean, it's not like the old media died in a day. The blogging phenomena started, and writers/etc. were able to promote their work directly in ways that they hadn't managed to before. Margins on book sales went down, which has cut into promotional budgets (as much as anything). Also, thanks to developments in technology the size of most first runs is much smaller than it used to be. This is probably a good thing, but it also means that the capital investment on new authors and books is much less than it used to be. ...and the end result of this is that we're prone to seeing publishing companies as "Authors Services" companies.

As a model for "what comes next," services for authors is a huge part of what we need from the publishing companies. Centralizing and connecting authors with people who can provide big-picture editing, with people who do copy editing and proof reading, with people who do the cover art and layout of the book itself, and with people who can do the promotional work, getting the book plugged into the distribution channels. These are real needs, that aren't going to evaporate any time soon.

But what about the editorial and curatorial roles of publishers? What about the branding associated with publishing houses? I think there's probably some future for critical discourse in blogs and in digital forums, which will provide some of these functions, but that's not the full answer, and I'm not sure what the full answer is.

As I wrote, earlier, I think figuring out some sort of subscription system to support content creation and distribution. I think having the economic superstructure in place, or at least worked out conceptually is really important before we start working on new technologies, like ebook readers, and digital content distribution channels.

It's an interesting time to be around, that's for sure.

The Dangers of Consolidation

I mentioned in an earlier post that I thought Barnes and Noble was largely responsible for the ongoing and impending collapse of the publishing industry, and that's just the sort of thing that I couldn't leave a lone without a little bit of further pondering.

The assertion is that Barnes and Nobel, and Borders particularly as they competed for near total domination of the local-book retail market, forced a consolidation of the publishing industry at the very moment when the worst possible thing for publishing was consolidation.

Consolidation allows an operation to make a bunch of money quickly. The mechanics of this are pretty simple, after all. When yo consolidate you can cut all sorts of mundane expenses, from the physical costs of maintaining parallel operations, to hard costs like printing and shipping costs that can benefit from collective organization.

Amazon had a role in the consolidation of sale of books, certainly, but Amazon has always been a distribution and data company, primarily. Their strategy is to find a way to turn a profit on the sale of goods, any goods: they do this by having a complete inventory of everything and levering a lot of data concerning buying habits and browsing habits to make sure people who are shopping find something to buy.

Where Amazon's limiting factor is connecting people who want to buy things (books) with books they might like to buy, the "traditional" book sellers, are limited by the amount of shelf space they can use to display and promote books. So they edged all of the little booksellers out of business by having huge stores and coffee shops and so forth, and then faced with too many books and not enough shelf space, they used their muscle to push the publishing industry toward increased consolidation and a "blockbuster" business model.

Blockbusters are how the movie industry works. Production companies make a bunch of movies, on the premise that if one or two turn a huge profit, they can afford to make a number of movies that flop or that just break-even. Hence the great power of reliable successes: another John Grisham novel, Return of the Mummy King VI, etc., the "copy-cat" phenomena and the erosion of the independent movie production business.

Book sellers were culpable as well--consolidation is attractive in what are essentially commodity businesses--and selling easily produced paper-based volumes is a commodity business. Maybe it would have happened anyway, but the undeniable market success of Barnes and Nobel is not, given what I can tell from where I'm sitting, a marker of success of the publishing industry as a whole.

And you know, when you're a book seller, throwing the publishing industry onto the tracks before an oncoming train, to achieve some mildly impressive profits for a decade seems... not incredibly bright. And not the kind of thing that I'm interested in supporting or putting my faith in.

It strikes me that this "consolidation meme" is a common feature of unsustainable and inauthentic economy, and it extends beyond book retail into other failed and failing sectors of the ecomony.

  • Banks. Obvious here. The big banks lost track of the micro economics that make the macro economics go, and we got things like sub-prime morgages, because while they make sense from the consolidated-bank perspective they don't make sense to people. Like, the John Grisham-esque Legal/Drama/Thriller book makes a lot of sense to the booksellers and the publishers, but most people can only really read so many of them before loosing interest.
  • Software. Microsoft's production of windows makes a lot of sense if you're a big company, but if you use computers in a specialized way, Windows is like an illfitting suit from Target. It works, but it's uncomfortable and rough around the edges. There is general consensus that "The Microsoft Way" isn't the best technological solution to the various problem, even among people who use it regularly (developer tools might require a slightly more complex investigation).
  • Your Example Here. Leave a note in the comments.

Thoughts?

Wordpress Limitations

Wordpress is great software, and I've been a user for many years. Many years. It used to be called "b2" and I used it then as well. There are a lot of more powerful content management systems, a lot of systems that are much more flexible than wordpress these days, and often I get the feeling that other platforms attempt to define themselves in contrast to wordpress. In the larger sense, this post is an attempt to resist this temptation while also exploring the limitations of wordpress.

Wordpress is a pure blogging engine: it provides interfaces for writers to publish weblogs (blogs), manage content (to some degree) and generate pages based on templates. Before wordpress, blogging was done either by hand edited text files, or by systems that complied static HTML from some sort of database. [1] Wordpress is an improvement because it's easy to install, it's reliable, and pages generate dynamically on viewing, rather than just when the site owner hits "save" or "rebuild." In the end, we discovered that systems where managing "websites" was divorced from (even simple) server management had a great democratizing effect on content, and that's sort of the core of wordpress.

Because wordpress is designed to be a blogging platform, it doesn't need to be as flexible as other generalized content management systems. Flexibility comes at the cost of complexity, and developers decided that in some cases, less was, in fact, more. There are a lot of things that you could do with b2 (albeit with some hacking) because the site generation/templating system was much less rigid, at the same time, it was much easier to get sites with broken links, and bad pages, particularly as you changed from theme to theme. That's bad, and it seems pretty reasonable to me to want to avoid that.

The end result is a program that does almost everything you could want it to do as long as you only want a blog, if you try and stretch it too far it simply won't work. Well it will work, but the advantage of using Wordpress to manage a website that isn't a blog (or very similar to one) disappears quickly when you have to impose informal limitations on how you enter content in the system to generate well formated pages. It's a slippery slope, and you'd be surprised how quickly a site goes from being a standard Wordpress site, to requiring customized themes, specialized content entry patterns. And pretty soon, a lot of the things that make Wordpress "simple" and "essay," aren't really available to your new site. That's the limitation of Wordpress.

Knowing where the line is, is often the largest challenge in Wordpress development, and being able to say, "you know, this is the kind of site that you really want to be building with Django, or Drupal, or Rails, or Expression Engine," Or even saying "you know this is the kind of site that we could probably do more effectively using flat files and PHP includes. Wordpress is great, and in the cases where it's well suited to the task at hand, it's the ideal solution. In other situations? Less so.

Onward and Upward!

[1]Interestingly, this whole "static site compiling" is making a come back, because it turns out that dynamic page generation doesn't scale as well as we thought it would five or six years ago. So we have static site compilers and complex caching tools. What comes around, goes around I guess.

SEO Nonsense

I read the following phrase on my travels this past week: "we'll just have to wait till the SEO does it's thing." This is sort of a typical phrase that gets throw around on the "commercial internet," and it wasn't out of place. Indeed, I think all the readers of the article probably understood what the author was trying to convey. But it struck me as sort of odd. Here's why:

It's a completely empty statement. SEO (Search Engine Optimization) refers to the collection of techniques that are used to "raise" a given web page's ranking in search results. Because there isn't a hell of a lot of competition in this market, basically this amounts to trying to "game" Google.

Which... is sort of a loosing proposition. Google's algorithms (or the key components) are top secret, and what we do know about how google arranges searches is that the more pages link to a given page, the more favorably Google's algorithm's view that pages, this lets Google's search results reflect a sort of emergent semantic organization of the world wide web. This means that when we search google, more often than not, we're mostly searching the most interreferential pages on the internet.

It's true that there are a lot of sites that don't have a lot of "juice" in Google, and that's really frustrating for people who create websites, but Google's domination of the internet-search marketplace is due largely to the quality of results that this reference-based system plays.

And in light of this, I hope it's pretty obvious that SEO is mostly a crock of shit. You can't game Google, and more to the point you don't want to. Though I think the prevalence of SEO an interesting admission for the "commercial internet" that traditional advertising-based marketing models has utterly failed on the internet.

To my mind the ideological parent of SEO was "search engine submission" services, which would purportedly "submit" your website to search engines so that people could find your site. For a fee, usually. Clearly this didn't work, because the return was so diffuse, and because no one really wants to use a search tool where the results are based on "submissions" which are paid for by the content producers. There's a reason why most of us use Google and not AltaVista, AskJeeves, Excite, Lycos, Infoseek, and so forth.

Now having said that there are some things that you can do to encourage your site's ranking in google (ie. get people to link to you on their sites,) I'd call this "good writing," or "effective communication," or "best practices," not "SEO" but you know whatever works. Here's what I think really works.

1. Be interesting, and have something to say. No one wants to read a website that's boring. That's why my readership is so low ;) but if you can't make the attempt, no amount of good mojo is going to help your site.

2. Post regularly. Really regularly. This resonates with this idea, the way to get good at doing 1., and make sure that people keep reading your site (and linking to you) is to provide dynamic and fresh content.

3. If you're a company, write not only about what you do and your clients, but also about what your clients are interested in. This might mean talking about and linking to your competitors, don't worry, rising tides raise all boats.

4. Participate in real life conversations. Most people learn about new websites via word of mouth connections formed in unusual contexts. There's a reason why most of the leaders of the independent web (bloggers) are either: in New York City, San Francisco, or have been going to SXSWi since the beginning. (There's also a minority of L.A. based bloggers). Talk to people, talk about your work, and talk to the other people who are creating content.

5. Write emails. This is the second stage of what starts in 4. Digital networking connections rest mostly on one-on-one email correspondences, and listserv conversations, despite all sorts of next wave technology like twitter, facebook, and linked in. Getting really good at writing quick, meaningful emails and staying on top of your correspondence is requisite.

6. Top load your content and titles. This falls under the category of good practice, and it mirrors the way newspaper columns are structured. Give as much information away as soon as possible, put all the details at the end, and write in a style that's simple and designed to be easily and quickly read. There's a lot out on the internet, and the the less time you take to make a specific point/joke/insight, the better.

7. Provide full RSS feeds, and don't put things behind "cut/fold" tags so that people have to click through to "read more." The former is good sense, and represents reaching out to other content producers (the people who read your site,) and the later is just good sense. [1] Other content producers are the people who have the real power over your search engine ranking, and making your content accessible is the first step in getting the content read.

8. Use a site design which maximizes readability and visibility, so that people can--you know--read your content, rather than marvel at your superior design capabilities.

Basically write a good site, network well, and don't waste your time on snake oil and chants. /end.

[1]The exception to this rule is live journal, as many people read LJ via the "Friend's Page" which induces a slightly different community standard. In general though, it provides yet another obstacle between a reader who would might read your, and in general your design/style should work to be more inclusive.