For the last, call it 8 years, having a website, dynamically generated with most of the content pulled on (nearly) every reload from a database. MySQL or PostgreSQL, or some such. It’s a great model, in a lot of ways, because it represents the (near) ultimate separation of content and display, it means that static pages don’t have to be generated whenever you generate new content. Dynamic, database driven websites were going to the future, and I think history has proven this argument.

At the same time, I think a further analysis is required.

Just to be super clear the advantages of database driven websites are that:

• Pages don’t have to be regenerated when content changes.
• Content is more customizable and can be pulled together in ad-hoc settings.

I would argue however, that there are some fundamental weaknesses that this system of deploying websites promotes:

1. Database driven websites increase the complexity of web-site software by a magnitude or two. I can in my sleep hack together a static website (most people can); working with database requires a much more specialized system that is harder for website owners to maintain. While separating content from display is often an effort to make systems easier to understand and change, in point of fact, databases make website maintenance a specialized task.
2. Database driven websites have a lot of overhead. Because pages need to be regenerated regularly, they require beefy hardware to work correctly. On top of this database systems need to be cached in such a way, that they’re not quite as dynamic as they once were.
3. Databases are mostly server-based technologies, which means a lot of the dynamic client-side scripting (EMCAscript/JavaScript and AJAX/AHAH) that are all the rage these days (and what people most often mean when they say “dynamic”) aren’t nearly as dependent on databases as what’s going on in that space.
4. Given the very structured nature of databases, websites often need to develop their content structure with near prescience regarding what’s going to happen in their site for the next five years. This is complicated, difficult, and often means that the same content-system needs to be redeveloped (often at great cost) far too often.

In light of this I’ve been thinking, increasingly, that the future of websites will likely be powered by a much different kind of website software. Here are some observations about the future of the web:

• Structured data formats, and plain text files are the future. Stored in/with formats like yaml, we can (very easily) have flexible structures can adapt to the changing needs of a website owner.
• Some very large sites (eg. facebook, wikipedia) will likely always be powered by databases, because in situations where a single website has > 100,000 pieces of content databases begin to make sense. Remarkably, single websites so rarely have that much content. Particularly if engineered correctly.
• Most content on the web doesn’t change very often. We regenerate pages thousands and thousands of times a day that would be unlikely to be update more than a dozen times a day.

This is not to say that there aren’t several challenges to the prospect of websites powered by static-site generators/compilers. They are:

• Some content will likely always be compiled using some very basic server-side includes, and dynamic content will continue to be generated using java script or something similar.
• Authentication and web-based security will likely also need to be built into webservers directly (the direction, I think things are going in anyway) and complicated access control (for intranets, say) may still require databases.
• Web-based interfaces for editing (ubiquitous, in page, “edit this” links). and commenting systems often need more dynamic functionality than other content. We need to both figure out a way to do commenting/annotation in a static-based system and find a way to do commenting in a more socially productive manner (existing locally hosted commenting systems, are I think fundamentally broken).
• Concurrent Editing. Wiki engines address this to some degree, and I think we need additional productive ways of addressing this in a truly user friendly manner, that doesn’t rely on over powered databases for what is probably an edge case.

Thoughts? I’m not describing technology that doesn’t exist, but I am suggesting that the current “way of doing things,” isn’t the future of how we will “do content on the web.” The tools are out there, all that’s missing is a simple, user friendly, method for pulling all this content together.

I’m really interested in seeing what people are coming up with.

In my post about my fact file I said that I was going to “try things out and see how it goes” before I posted code to see how things work in the system. Well, I think things are pretty stable (I haven’t tweaked much), so I’m going to post my remember template, for the system described in that post.

(setq org-remember-templates
   '(("data" ?d "* %^{Title} %^g \n :PROPERTIES:\n :date:
       %^t\n :cit e-key: %^{cite-key}\n :link: %^{link}\n
       :END:\n\n %x %?"  "~/org/data.org")))`

If you want to tweak further, check out the relevant section of the org manual.

Enjoy and collect facts with abandon!

When Jaiku gave microbloggers (ie users of services like identi.ca twitter, etc.) the ability to aggregate their content onto Jaiku from blogs, flickr, delicious, and of course twitter as well, I noted that it wouldn’t take much for the Jaiku to become a ghost town, where no one would have to actually create new content or read in order for new content to filter through the site. While syndication allows people to do some really amazing things with content, without filtering, context, and moderation it’s really easy to become overwhelmed. Frankly Jaiku provide(s/ed) enough filtering ability to make this useful but this “problem” continues:

There are services like ping.fm which let you type once and send widely to a whole host of audiences on a bunch of different websites. Which is great: it lets you be present without taking up all your time, it allows you to reach a very fractured audience. Good stuff. Right?

Except not so much. The thing about “social media” and these new very conversational media is that they’re not so much about sending widely. They’re great for reaching people, in the moment, but they’re really more about having conversations with a community.

So we need something better. Because being shut off from your friends because you use different websites is bad; but what amounts to using AM radio cranked up loud to talk to your friends in a couple of different coffee houses is worse. I think open micro blogging (like identi.ca and laconica) is a step in the right direction, better independent profiles and curatorial tools would be another.

Thoughts?

I commented to someone, probably Caroline, that I found cooking to be a lot like writing fiction. You start out with a vague I dea of what you’re trying to do, some really firm building blocks (ingredients; characters/settings), some rather impressive limiting factors (time, physics, literary conventions), and a number of equally good options (narrative voice, baking, chapter organization, broiling, pacing, frying) and then you get on this roller coaster where, despite whatever training you have, and formal knowledge about how things should work, you mostly just pay attention to your gut and pray that you don’t make a huge mess of things as you’re shepherding your lunch/novel from the refrigerator/mind to your stomach/readers.

Needless to say, I rather enjoy cooking.

I have a few tendencies in cooking that determines my method: I generally like to cook a lot once, maybe twice, a week and then reheat and reconfigure whatever it is that I made again and again. So I’m a fan of soups, stir-frys, roasted veggies, vegetable/pasta salads and pestos, and things like that. I’m also prone to just cooking up, say, a lot of zuchini, and then making omlets/pasta/etc. with it throughout the week (without lots extra fuss.)

I should also add a few things. First I’m not a vegetarian, really. I don’t mind eating/cooking meat, but I do so with a lot of moderation, and while there are some vegetables that I don’t really understand the appeal of (celery? wtf. beets? feh), I’ve found that my use of meat in meals is often pretty minimal. My natural tendency, having said all this, is to eat quick things (tuna fish with mayo; noodles; etc.) rather than actually have to cook at all, but sometimes it’s good to resist this tendency.

As part of my tenure helping my grandmother out, I’ve been doing a lot of cooking, so I’ve been doing a lot of this, and I think it’s worked pretty well. Last week I was cooking. I had bought some eggplant which I wanted to roast and make a sort of eggplant/tomato/mushroom sauce to serve over rice. Roast eggplant, mix in tomato sauce, with some red peppers and mushrooms, sounds good?

Like good novels/meals, it got into the pan and decided to be something totally different. On the upside, it was amazing. Here’s a brief recipe/guideline for what I made:

Ingredients

• Garlic, Fresh, in great quantity. Cut cloves in half.
• 1 Can of artichoke hearts, drained and quartered.
• 1 can of straw mushrooms, drained.
• Some amount of beef sausage (12 oz?) sliced (optional)
• 1 or 2 red peppers, sliced
• 2 or 3 Zucchini, sliced
• 1 sweet onion, chopped
• 2 - 3 Eggplant roasted with skin and seeds removed (cut in half, spray/apply a minimal amount of olive oil, and sail for roasting, when the skin is blackened and separated peel it, remove seeds and chop the remaining eggplant.)
• A bit of olive oil
• Rice vinegar (though other white vinegars should work)
• Rice (I used sushi rice, but that’s personal preference.)

Preparation

Get all of the ingredients ready. Eggplant roasting can happen earlier. Once everything starts cooking it goes pretty fast.

Begin by heating the oil with garlic and onion and sausage until they begin to brown, add zucchini and red pepper and cook until the zuchini gives sings of browning. It might take 15 minutes to get to this point.

Add Eggplant, mushrooms, and artichoke hearts, cook for ~3-5 minutes, add 3/4 cup of vinegar (or more depending on how much food you’re making. You can use wine if that’s your speed.) Turn heat down slightly and simmer for 3-5 minutes.

Use your judgment, my times are really estimates/guesses.

Serve with rice and enjoy.

So I was hoping to write you this morning from a brand new blogging system powered by jekyll that I did a little bit of tweeking on. Unfortunatly, all the pieces aren’t all in place, yet. So tychoish is still a wordpress event still on wordpress. The highlights:

• It seems to have not worked, because it’s notion of categories isn’t a bit more firm than my own notion of categories (ie. coda and tychoish), and there isn’t any clever (or not so clever) way to get around this. It’s way more geeky than normal, but basically there’s no way (that I can figure) without categories to get an array/object of some segment of the posts without using jeykll’s categories.

  The categories that jekyll has are really powerful, and my problem isn’t so much how they work, as it is that I have 1300-ish posts written “in the old way” that I need to figure something out for. I’ll do that, but it’ll take a bit longer, and it’ll mean that there’ll be a more substantive update to the site.

• I did something that I think improves jekyll, by adding an aditional permalink format to my fork. I’m not even passingly familar with Ruby (and I may have a slight professed disdain for it) and while it wasn’t a big change nor did it require a bunch of logic/smarts, it works and I think I did it “right” for whatever that means.

  I also, in the course of this, got sed to work right, which was mostly just a case of me reading the manual wrong for years and then avoiding it. But personal victories are victories all the same.

I can explain my jekyll issue/need in more detail if anyone wants to help me hack on this. Cheers!

My grandmother watches the evening news, which isn’t particularly noteworthy except that it’s not something that I do unless I’m near someone who does. I tend to get my news via the radio, and the Internet, and while the visuals are instructive, it’s not something that I’ve had a lot of experience watching in recent years.

Which is why I was surprised by how bad they’ve gotten in the past few years. Not only is the news shorter, but there’s less variation in the coverage (all the networks/shows cover the same half dozen stories, with much the same slant.) And there’s this pervasive inline advertising thing which I swear is a new phenomena. It’s unsettling at the very least.

ps. I also correctly diagnosed a case of spasmodic disphonia in they junior senator from main on the television.

There’s a lot of content on the Internet. This as pretty much always been the case, but it’s especially true these days. There’s so much content that it’s difficult to comprehend the amount of content on this (very small) website, let alone the content of “all the blogs” or other kinds of sites.

In the early days of blogging, the largest archives included a hundred or even two hundred posts, no more than a year or two of archives, and if you read the dozen or two dozen blogs in the general network that you covered, you could be pretty sure that you were reading some significant portion of the “weblog discourse” and all the same blogs that the people writing the blogs were reading. When you found a new blog, the chance is that you could read the entire archive in an hour or two if you were so inclined.

It’s a different world now. Blogs have many years of archives, hundreds upon hundreds of posts, what we read is only a drop in the bucket of whats out there.

Success, (a relative concept indeed) as content creators on the Internet, in this saturated market requires a very different strategy. Pumping out more content, just further saturates our websites, we need some other approach. It doesn’t help that the publication systems we use to power content on the web (wordpress, drupal, blogger) are designed for an earlier era.

I would propose that success in the next era will revolve around, individuals and technological solutions that make “curation” easier and more effective. Curation you ask? In museums we see the display of only a fraction of the material held in the collection. Likely 10% or less. What you do see is selected (regularly) by curators who know and manage the collection, and take those materials and display them in such a way as to convey some sort of broader message about the subject of an exhibit.

The same thing can (and should) happen on the web, with content. We need content producers to go through their writing and say “read this first for an introduction to my work/ideas,” we need to connect content producers with curators who synthesize posts (with links) to great content that’s “out there” (like tumble logs), we to integrate tools and structures that make curation easier in existing content-management software.

The truth is that many of these things are starting to happen: I’ve seen some bloggers who attempt to filter their posts for their readers (and I’m going to be doing more of this,) tumble logs, when well done are curatorial. The software is here in bits and pieces, but the truth is that curation has to be a mostly manual task (which is the source of its value), so I think it’ll be a while before we work out the kinks so that the software facilitates this kind of work. In the mean time…

Work as if you Live in the Early Days of a Better Nation.

I’ve been writing a great deal in the past few days: blog posts, the seed content for a wiki, fiction at a somewhat impressive pace (for me). While (at press time) there are still lots of things on my plate and storms on the horizon it’s very true that doing creative things, getting work done makes it considerably easier to do more creative things, and to get more work done. The stuff on my plate and the storms on the horizon in another seem less threatening. It’s not logical, but I’m not going to argue with that.

Of note: I posted another story to Critical Futures from Chapter 6 of Knowing Mars, this is the last scene in this chapter, and I’ll post it in three chapters. After that, we’re going to get PDFs of Chapters 4-6, and I’ll be posting more about that in time. Thanks to everyone’s replies to my post about my emacs process, both on the thread and on identi.ca. I’m much impressed with all of your wisdom, and assistance. I didn’t know about wdired-mode which I shall explore, and while I did know about magit, I hadn’t gotten around to trying it yet. Here I come.

One last thing, if there are any perl, ikiwiki, CGI::FormBuilder, Dreamhost wizards out there, I would be very flattered and indebted if you’d take a look at this bug report I filed on ikiwiki, I’m having some sort of minor problem with CGI form generation, which is such a minor bug that I’m highly annoyed at the concept. On the one hand, it’s a lot of effort to get ikiwiki working under these conditions, on the other hand it doesn’t make a lot of sense to pay for a really robust server (with root access) for what will really be a minimal amount of content. Anyway, your help would be much appreciated.

ps. sorry for the late content.

pps. if you record a podcast over a skype (or other VoIP service) with Audacity and you’re running Ubuntu/Debian/Linux… I’d love to hear from you.